Innovative Mobile and Internet Services in Ubiquitous Computing: Proceedings of the 16th International Conference on Innovative Mobile and Internet ... (Lecture Notes in Networks and Systems) 3031088182, 9783031088186

Citation preview

Lecture Notes in Networks and Systems 496

Leonard Barolli Editor

Innovative Mobile and Internet Services in Ubiquitous Computing Proceedings of the 16th International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing (IMIS-2022)

Lecture Notes in Networks and Systems Volume 496

Series Editor Janusz Kacprzyk, Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland Advisory Editors Fernando Gomide, Department of Computer Engineering and Automation—DCA, School of Electrical and Computer Engineering—FEEC, University of Campinas— UNICAMP, São Paulo, Brazil Okyay Kaynak, Department of Electrical and Electronic Engineering, Bogazici University, Istanbul, Turkey Derong Liu, Department of Electrical and Computer Engineering, University of Illinois at Chicago, Chicago, USA Institute of Automation, Chinese Academy of Sciences, Beijing, China Witold Pedrycz, Department of Electrical and Computer Engineering, University of Alberta, Alberta, Canada Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland Marios M. Polycarpou, Department of Electrical and Computer Engineering, KIOS Research Center for Intelligent Systems and Networks, University of Cyprus, Nicosia, Cyprus Imre J. Rudas, Óbuda University, Budapest, Hungary Jun Wang, Department of Computer Science, City University of Hong Kong, Kowloon, Hong Kong

The series “Lecture Notes in Networks and Systems” publishes the latest developments in Networks and Systems—quickly, informally and with high quality. Original research reported in proceedings and post-proceedings represents the core of LNNS. Volumes published in LNNS embrace all aspects and subfields of, as well as new challenges in, Networks and Systems. The series contains proceedings and edited volumes in systems and networks, spanning the areas of Cyber-Physical Systems, Autonomous Systems, Sensor Networks, Control Systems, Energy Systems, Automotive Systems, Biological Systems, Vehicular Networking and Connected Vehicles, Aerospace Systems, Automation, Manufacturing, Smart Grids, Nonlinear Systems, Power Systems, Robotics, Social Systems, Economic Systems and other. Of particular value to both the contributors and the readership are the short publication timeframe and the world-wide distribution and exposure which enable both a wide and rapid dissemination of research output. The series covers the theory, applications, and perspectives on the state of the art and future developments relevant to systems and networks, decision making, control, complex processes and related areas, as embedded in the fields of interdisciplinary and applied sciences, engineering, computer science, physics, economics, social, and life sciences, as well as the paradigms and methodologies behind them. Indexed by SCOPUS, INSPEC, WTI Frankfurt eG, zbMATH, SCImago. All books published in the series are submitted for consideration in Web of Science. For proposals from Asia please contact Aninda Bose ([email protected]).

More information about this series at https://link.springer.com/bookseries/15179

Leonard Barolli Editor

Innovative Mobile and Internet Services in Ubiquitous Computing Proceedings of the 16th International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing (IMIS-2022)

123

Editor Leonard Barolli Department of Information and Communication Engineering Fukuoka Institute of Technology Fukuoka, Japan

ISSN 2367-3370 ISSN 2367-3389 (electronic) Lecture Notes in Networks and Systems ISBN 978-3-031-08818-6 ISBN 978-3-031-08819-3 (eBook) https://doi.org/10.1007/978-3-031-08819-3 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Welcome Message of IMIS-2022 International Conference Organizers

Welcome to the 16th International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing (IMIS-2022), which will be from June 29 to July 1, 2022, in conjunction with the 16th International Conference on Complex, Intelligent and Software Intensive Systems (CISIS-2022). This International Conference focuses on the challenges and solutions for Ubiquitous and Pervasive Computing (UPC) with an emphasis on innovative, mobile and Internet services. With the proliferation of wireless technologies and electronic devices, there is a fast growing interest in UPC. UPC enables to create a human-oriented computing environment where computer chips are embedded in everyday objects and interact with physical world. Through UPC, people can get online even while moving around, thus having almost permanent access to their preferred services. With a great potential to revolutionize our lives, UPC also poses new research challenges. The conference provides an opportunity for academic and industry professionals to discuss the latest issues and progress in the area of UPC. For IMIS-2022, we received many paper submissions from all over the world. The papers included in the proceedings cover important aspects from UPC research domain. We are very proud and honored to have two distinguished keynote talks by Prof. Keita Matsuo, Fukuoka Institute of Technology, Japan, and Dr. Anne Kayem, Hasso-Plattner-Institute, University of Potsdam, Germany, who will present their recent work and will give new insights and ideas to the conference participants. The organization of an International Conference requires the support and help of many people. A lot of people have helped and worked hard to produce a successful IMIS-2022 technical program and conference proceedings. First, we would like to thank all the authors for submitting their papers, the Program Committee Members, and the reviewers who carried out the most difficult work by carefully evaluating the submitted papers. We are grateful to Honorary Chair Prof. Makoto Takizawa, Hosei University, Japan, for his guidance and advice.

v

vi

Welcome Message of IMIS-2022 International Conference Organizers

Finally, we would like to thank Web Administrator Co-Chairs for their excellent and timely work.We hope that all of you enjoy IMIS-2022 and find this a productive opportunity to learn, exchange ideas and make new contacts.

IMIS-2022 Organizing Committee

Honorary Chair Makoto Takizawa

Hosei University, Japan

General Co-chairs Hsing-Chung Chen Baojiang Cui

Asia University, Taiwan Beijing University of Posts and Telecommunications, China

Program Committee Co-chairs Hyunhee Park Lidia Ogiela

Myongji University, Korea AGH University of Science and Technology, Poland

Advisory Committee Members Vincenzo Loia Arjan Durresi Kouichi Sakurai

University of Salerno, Italy IUPUI, USA Kyushu University, Japan

Award Co-chairs Tomoya Enokido Hae-Duck Joshua Jeong Fang-Yie Leu

Rissho University, Japan Korean Bible University, Korea Tunghai University, Taiwan

vii

viii

IMIS-2022 Organizing Committee

International Liaison Co-chairs Marek Ogiela Elis Kulla Farookh Hussain

AGH University of Science and Technology, Poland Fukuoka Institute of Technology, Japan University of Technology Sydney, Australia

Publicity Co-chairs Kangbin Yim Hiroaki Kikuchi Keita Matsuo

Soonchunhyang University, Korea Meiji University, Japan Fukuoka Institute of Technology, Japan

Finance Chair Makoto Ikeda

Fukuoka Institute of Technology, Japan

Local Arrangement Co-chairs Tomoyuki Ishida Kevin Bylykbashi

Fukuoka Institute of Technology, Japan Fukuoka Institute of Technology, Japan

Web Administrators Phudit Ampririt Ermioni Qafzezi

Fukuoka Institute of Technology, Japan Fukuoka Institute of Technology, Japan

Steering Committee Chair Leonard Barolli

Fukuoka Institute of Technology, Japan

Track Areas and PC Members 1. Multimedia and Web Computing Track Co-chairs Chi-Yi Lin Tomoyuki Ishida

Tamkang University, Taiwan Fukuoka Institute of Technology, Japan

PC Members Noriki Uchida Tetsuro Ogi

Fukuoka Institute of Technology, Japan Keio University, Japan

IMIS-2022 Organizing Committee

Yasuo Ebara Hideo Miyachi Kaoru Sugita Akio Doi Chang-Hong Lin Chia-Mu Yu Ching-Ting Tu Shih-Hao Chang

ix

Osaka Electro-Communication University, Japan Tokyo City University, Japan Fukuoka Institute of Technology, Japan Iwate Prefectural University, Japan National Taiwan University of Science and Technology, Taiwan National Chung Hsing University, Taiwan National Chung Hsing University, Taiwan Tamkang University, Taiwan

2. Data Management and Big Data Track Co-chairs Been-Chian Chien Akimitsu Kanzaki Wen-Yang Lin

National University of Tainan, Taiwan Shimane University, Japan National University of Kaohsiung, Taiwan

PC Members Hideyuki Kawashima Tomoki Yoshihisa Pruet Boonma Masato Shirai Bao-Rong Chang Rung-Ching Chen Mong-Fong Horng Nik Bessis James Tan Kun-Ta Chuang Jerry Chun-Wei Lin

Keio University, Japan Osaka University, Japan Chiang Mai University, Thailand Shimane University, Japan National University of Kaohsiung, Taiwan Chaoyang University of Technology, Taiwan National Kaohsiung University of Applied Sciences, Taiwan Edge Hill University, UK SIM University, Singapore National Cheng Kung University, Taiwan Harbin Institute of Technology, China

3. Security, Trust and Privacy Track Co-chairs Tianhan Gao Lidia Ogiela Arcangelo Castiglione

Northeastern University, China AGH University of Science and Technology, Poland University of Salerno, Italy

PC Members Qingshan Li Zhenhua Tan

Peking University, China Northeastern University, China

x

Zhi Guan Nan Guo Xibin Zhao Cristina Alcaraz Massimo Cafaro Giuseppe Cattaneo Zhide Chen Clara Maria Richard Hill Dong Seong Kim Victor Malyshkin Barbara Masucci Arcangelo Castiglione Xiaofei Xing Mauro Iacono Joan Melià Jordi Casas Jordi Herrera Antoni Martínez Francesc Sebé

IMIS-2022 Organizing Committee

Peking University, China Northeastern University, China Tsinghua University, China Universidad de Málaga, Spain University of Salento, Italy University of Salerno, Italy Fujian Normal University, China Colombini, University of Milan, Italy University of Derby, UK University of Canterbury, New Zealand Russian Academy of Sciences, Russia University of Salerno, Italy University of Salerno, Italy Guangzhou University, China Second University of Naples, Italy Universitat Oberta de Catalunya, Spain Universitat Oberta de Catalunya, Spain Universitat Autònoma de Barcelona, Spain Universitat Rovira i Virgili, Spain Universitat de Lleida, Spain

4. Energy Aware and Pervasive Systems Track Co-chairs Chi Lin Elis Kulla

Dalian University of Technology, China Fukuoka Institute of Technology, Japan

PC Members Jiankang Ren Qiang Lin Peng Chen Tomoya Enokido Makoto Takizawa Oda Tetsuya Admir Barolli Makoto Ikeda Keita Matsuo

Dalian University of Technology, China Dalian University of Technology, China Dalian University of Technology, China Rissho University, Japan Hosei University, Japan Okayama University of Science, Japan Aleksander Moisiu University of Durres, Albania Fukuoka Institute of Technology, Japan Fukuoka Institute of Technology, Japan

IMIS-2022 Organizing Committee

xi

Track 5. Modeling, Simulation and Performance Evaluation Track Co-chairs Tetsuya Shigeyasu Bhed Bahadur Bista Remy Dupas

Prefectural University of Hiroshima, Japan Iwate Prefectural University, Japan University of Bordeaux, France

PC Members Jiahong Wang Shigetomo Kimura Chotipat Pornavalai Danda B. Rawat Gongjun Yan Akio Koyama Sachin Shetty

Iwate Prefectural University, Japan University of Tsukuba, Japan King Mongkut’s Institute of Technology Ladkrabang, Thailand Howard University, USA University of Southern Indiana, USA Yamagata University, Japan Old Dominion University, USA

6. Wireless and Mobile Networks Track Co-chairs Luigi Catuogno Hwamin Lee

University of Salerno, Italy Soonchunhyang University, Korea

PC Members Aniello Del Sorbo Clemente Galdi Stefano Turchi Ermelindo Mauriello Gianluca Roscigno Dae Won Lee Jong Hyuk Lee Sung Ho Chin Ji Su Park Jaehwa Chung Massimo Ficco Jeng-Wei Lin

Orange Labs—Orange Innovation, UK University of Naples “Federico II”, Italy University of Florence, Italy Deloitte Spa, Italy University of Salerno, Italy Seokyoung University, Korea Samsung Electronics, Korea LG Electronics, Korea Korea University, Korea Korea National Open University, Korea University of Campania “Luigi Vanvitelli”, Italy Tunghai University, Taiwan

xii

IMIS-2022 Organizing Committee

7. Intelligent Technologies and Applications Track Co-chairs Marek Ogiela Yong-Hwan Lee Jacek Kucharski

AGH University of Science and Technology, Poland Wonkwang University, Korea Technical University of Lodz, Poland

PC Members Gangman Yi Hoon Ko Urszula Ogiela Lidia Ogiela Libor Mesicek Rung-Ching Chen Mong-Fong Horng Bao-Rong Chang Shingo Otsuka Pruet Boonma Izwan Nizal Mohd Shaharanee

Gangneung-Wonju National University, Korea J. E. Purkinje University, Czech Republic AGH University of Science and Technology, Poland AGH University of Science and Technology, Poland J. E. Purkinje University, Czech Republic Chaoyang University of Technology, Taiwan National Kaohsiung University of Applied Sciences, Taiwan National University of Kaohsiung, Taiwan Kanagawa Institute of Technology, Japan Chiang Mai University, Thailand University Utara, Malaysia

8. Cloud Computing and Service-Oriented Applications Track Co-chairs Baojiang Ciu Neil Yen Flora Amato

Beijing University of Posts and Telecommunications, China The University of Aizu, Japan University of Naples “Frederico II”, Italy

PC Members Aniello Castiglione Ashiq Anjum Beniamino Di Martino Gang Wang Shaozhang Niu Jianxin Wang

University of Naples Parthenope, Italy University of Derby, UK University of Campania “Luigi Vanvitelli”, Italy Nankai University, China Beijing University of Posts and Telecommunications, China Beijing Forestry University, China

IMIS-2022 Organizing Committee

Jie Cheng Shaoyin Cheng Jingling Zhao Qing Liao Xiaohui Li Chunhong Liu Yan Zhang Hassan Althobaiti Bahjat Fakieh Jason Hung Frank Lai

xiii

Shandong University, China University of Science and Technology of China, China Beijing University of Posts and Telecommunications, China Beijing University of Posts and Telecommunications, China Wuhan University of Science and Technology, China Heinan Normal University, China Yan Hubei University, China Umm Al-Qura University, Saudi Arabia King Abdulaziz University, Saudi Arabia National Taichung University of Science and Technology, Taiwan University of Aizu, Japan

9. Ontology and Semantic Web Track Co-chairs Alba Amato Fong-Hao Liu Giovanni Cozzolino

Italian National Research Council, Italy National Defense University, Taiwan University of Naples “Frederico II”, Italy

PC Members Flora Amato Claudia Di Napoli Salvatore Venticinque Marco Scialdone Wei-Tsong Lee Tin-Yu Wu Liang-Chu Chen Omar Khadeer Hussain Salem Alkhalaf Osama Alfarraj Thamer AlHussain Mukesh Prasad

University of Naples “Federico II”, Italy Italian National Research Center (CNR), Italy University of Campania “Luigi Vanvitelli”, Italy University of Campania “Luigi Vanvitelli”, Italy Tam-Kang University, Taiwan National Ilan University, Taiwan National Defense University, Taiwan University of New South Wales (UNSW) Canberra, Australia Qassim University, Saudi Arabia King Saud University, Saudi Arabia Saudi Electronic University, Saudi Arabia University of Technology Sydney, Australia

xiv

IMIS-2022 Organizing Committee

10. IoT and Social Networking Track Co-chairs Sajal Mukhopadhyay Francesco Moscato

National Institute of Technology, Durgapur, India University of Campania “Luigi Vanvitelli”, Italy

PC Members Animesh Dutta Sujoy Saha Jaydeep Howlader Mansaf Alam Kashish Ara Shakil Makoto Ikeda Elis Kulla Shinji Sakamoto Evjola Spaho

NIT Durgapur, India NIT Durgapur, India NIT Durgapur, India Jamia Millia Islamia, New Delhi, India Jamia Hamadard, New Delhi, India Fukuoka Institute of Technology, Japan Fukuoka Institute of Technology, Japan, Japan Kanazawa Institute of Technology, Japan Polytecnic University of Tirana, Albania

11. Embedded Systems and Wearable Computers Track Co-chairs Jiankang Ren Keita Matsuo Kangbin Yim

Dalian University of Technology, China Fukuoka Institute of Technology, Japan SCH University, Korea

PC Members Yong Xie Xiulong Liu Shaobo Zhang Kun Wang Fangmin Sun Kaoru Sugita Tomoyuki Ishida Noriyasu Yamamoto Nan Guo

Xiamen University of Technology, Xiamen, China The Hong Kong Polytechnic University, Hong Kong Hunan University of Science and Technology, China Liaoning Police Academy, China Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, China Fukuoka Institute of Technology, Japan Fukuoka Institute of Technology, Japan Fukuoka Institute of Technology, Japan Northeastern University, China

IMIS-2022 Organizing Committee

xv

IMIS-2022 Reviewers Leonard Barolli Makoto Takizawa Fatos Xhafa Isaac Woungang Hyunhee Park Fang-Yie Leu Kangbin Yim Marek Ogiela Makoto Ikeda Keita Matsuo Francesco Palmieri Massimo Ficco Salvatore Venticinque Admir Barolli Elis Kulla Arjan Durresi Bhed Bista Hsing-Chung Chen Kin Fun Li Hiroaki Kikuchi Lidia Ogiela Nan Guo Hwamin Lee Tetsuya Shigeyasu Kosuke Takano Flora Amato Tomoya Enokido Minoru Uehara Tomoyuki Ishida Hwa Min Lee Jiyoung Lim

Tianhan Gao Farookh Hussain Omar Hussain Nadeem Javaid Chi-Yi Lin Luigi Catuogno Akimitsu Kanzaki Wen-Yang Lin Tomoki Yoshihisa Masaki Kohana Hiroki Sakaji Baojiang Cui Takamichi Saito Arcangelo Castiglione Shinji Sakamoto Massimo Cafaro Mauro Iacono Barbara Masucci Ray-I Chang Gianni D’Angelo Remy Dupas Aneta Poniszewska-Maranda Sajal Mukhopadhyay Tomoyuki Ishida Yong-Hwan Lee Lidia Ogiela Hiroshi Maeda Evjola Spaho Jacek Kucharski Yong-Hwan Lee Kevin Bylykbashi

IMIS-2022 Keynote Talks

Design and Implementation Issues of Omnidirectional Robots and Their Applications for Different Environments Keita Matsuo Fukuoka Institute of Technology, Fukuoka, Japan

Abstract. Intelligent robotic systems are becoming essential for increasing Quality of Life (QoL) and keeping health for growing population of elderly people. In our research, in order to solve human health problems and support elderly people, we consider the design and implementation of omnidirectional robots. In this talk, I will introduce our results to show how omnidirectional wheelchair robots can support people with disabilities at home and at workplace. In our work, we also consider the use of the omnidirectional wheelchair robots for playing tennis and badminton. I also will present the application of omnidirectional robot as a mesh router in Wireless Mesh Networks (WMNs) in order to provide a good communication environment.

xix

Is Privacy the Same as Security, or Are They Just Two Sides of the Same Coin? Anne Kayem Hasso-Plattner-Institute, University of Potsdam, Potsdam, Germany

Abstract. Almost every digital device either generates or consumes data in some form. The result is that the volumes of data collected grow exponentially each day. Data analytics proponents have mooted that it is now possible in some cases to actually predict future human behaviors based on data collected through tracking and various other means. On the other parallel, the question of privacy has become ever more important as users increasingly seek ways of guarding their personal data from exposure. This as such raises the question of what the distinction between privacy and security (data protection) is and what the boundary between the two should be. For instance, the 2014 incident of a hacker faking the German minister of defense’s fingerprints was considered to be a security breach. However, a closer look at this issue highlights the fact that distinguishing between whether or not this was a privacy breach that enabled a security breach, or vice versa, does not have a straightforward answer. In this talk, I aim to explain why in my view privacy is different from security and, while though both privacy and security are mutually interdependent, why it is important to make the distinction. The talk will be supported by various examples to characterize privacy and distinguish it from security. At the same time, I will also explain why the two concepts are in fact two sides of the same coin.

xxi

Contents

A Fuzzy-Based System for Assessment of Fog Computing Resources in SDN-VANETs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ermioni Qafzezi, Kevin Bylykbashi, Phudit Ampririt, Makoto Ikeda, Keita Matsuo, and Leonard Barolli Performance Evaluation of an AI-Based Safety Driving Support System for Detecting Distracted Driving . . . . . . . . . . . . . . . . . . . . . . . . Masahiro Miwata, Mitsuki Tsuneyoshi, Makoto Ikeda, and Leonard Barolli Autonomous Migration of Virtual Machines to Reduce Energy Consumption of Servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dilawaer Duolikun, Tomoya Enokido, Leonard Barolli, and Makoto Takizawa An Indicate System for Danger Detection and Its Soldering Motion Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tomoya Yasunaga, Tetsuya Oda, Kyohei Toyoshima, Yuki Nagai, Chihiro Yukawa, Kengo Katayama, and Leonard Barolli A Movement Adjustment Method for LiDAR Based Mobile Area Decision: Improving Control for AAV Mobility . . . . . . . . . . . . . . Nobuki Saito, Tetsuya Oda, Chihiro Yukawa, Kyohei Toyoshima, Aoto Hirata, and Leonard Barolli Design and Implementation of a Haptics Based Soldering Education System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kyohei Toyoshima, Tetsuya Oda, Tomoya Yasunaga, Chihiro Yukawa, Yuki Nagai, Nobuki Saito, and Leonard Barolli A Secure Data Sharing Scheme Based on CP-ABE in VANETs . . . . . . . Xinyang Deng, Tianhan Gao, Nan Guo, and Kang Xie

1

10

18

31

41

54

65

xxiii

xxiv

Contents

An Anonymous Authentication Scheme Based on Self-generated Pseudonym for VANETs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jiayu Qi and Tianhan Gao Wavelet Transform Based PID Sequence Analysis for IDS on CAN Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Md Rezanur Islam, Insu Oh, Munkhdelgerekh Batzorig, Myoungsu Kim, and Kangbin Yim Efficient CAN Dataset Collection Method for Accurate Security Threat Analysis on Vehicle Internal Network . . . . . . . . . . . . . . . . . . . . Yeji Koh, Seoyeon Kim, Yoonji Kim, Insu Oh, and Kangbin Yim

75

85

97

A New Method for Improving Throughput Performance by Simultaneous Transmission on Full-Duplex Wireless Communication Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 Hikari Hashimoto and Tetsuya Shigeyasu Intelligent Helmet Supporting Visually Impaired People Using Obstacle Detection and Communication Techniques . . . . . . . . . . . . . . . 120 Linh Thuy Thi Pham, Khoa Thanh Nguyen, Duyen Thuy Dao, Hai Thanh Nguyen, Huong Hoang Luong, and Nhan Trong Pham Van An Evacuation Support System for Promoting Distributed Evacuation in Evacuation Centers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 Tomoyuki Ishida and Ryosuke Nitama Evaluation on Noise Reduction in Subtitle Generator for Videos . . . . . . 140 Hai Thanh Nguyen, Tan Nguyen Lam Thanh, Tai Le Ngoc, Anh Duy Le, and Dien Thanh Tran Hierarchical Output Model of CNN Learning Using Multi Label Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 Jiha Kim, Agostinho António José, Jeena Kim, Yongho Kim, and Hyunhee Park Efficient Privacy-Preserving Authentication and Group Key Agreement Scheme in Fog-Enabled VANET . . . . . . . . . . . . . . . . . . . . . 161 Cong Zhao, Nan Guo, and Tianhan Gao 3D Reconstruction Based on the Depth Image: A Review . . . . . . . . . . . 172 Qingwei Mi and Tianhan Gao Mesh Routers Placement by WMN-PSODGA Simulation System Considering Stadium Distribution and RDVM: A Comparison Study for UNDX and UNDX-m Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184 Admir Barolli, Kevin Bylykbashi, Ermioni Qafzezi, Shinji Sakamoto, Leonard Barolli, and Makoto Takizawa

Contents

xxv

Unsupervised Deep Image Set Hashing for Efficient Multi-label Image Retrieval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196 Nan Guo, Cuixia Bai, and Yunxia Yang Hybrid Intelligence Approaches for Distributed Information Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207 Urszula Ogiela, Makoto Takizawa, and Lidia Ogiela MAC Layer Protocols for Underwater Acoustic Sensor Networks: A Survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 Elis Kulla, Keita Matsuo, and Leonard Barolli An Intelligent Robot Vision System for Recognizing Micro-roughness on Arbitrary Surfaces: Experimental Result for Different Methods . . . . 221 Chihiro Yukawa, Tetsuya Oda, Kyohei Toyoshima, Yuki Nagai, Tomoya Yasunaga, Chiaki Ueda, and Leonard Barolli Performance Evaluation of an Adaptive Anti-Packet Recovery Method Considering UAVs and Vehicles in an Urban Scenario . . . . . . . 230 Masaya Azuma, Shota Uchimura, Seiya Sako, Makoto Ikeda, and Leonard Barolli A Comparison Study of FC-RDVM and LDVM Router Placement Methods for WMNs by WMN-PSOHC Hybrid Intelligent System Considering Normal Distribution of Mesh Clients . . . . . . . . . . . . . . . . . 238 Shinji Sakamoto, Admir Barolli, Yi Liu, Elis Kulla, Leonard Barolli, and Makoto Takizawa Mask-Wearing Behavior Analysis by Using Expert Knowledge Acquisition Approach Under Covid-19 Situation . . . . . . . . . . . . . . . . . . 247 Hsing-Chung Chen, Yu-Lun Ho, and Shian-Shyong Tseng On the Conditional Pk -connectivity of Hypercube-Based Architectures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259 Tzu-Liang Kung and Yuan-Hsiang Teng Super K1;p -Connectivity of Locally Twisted Cubes . . . . . . . . . . . . . . . . . 267 Yuan-Hsiang Teng and Tzu-Liang Kung A Study on Image Transmission Based on Hopping LoRa . . . . . . . . . . . 275 Ching-Chuan Wei, Kuan-Chun Chang, and Chia-Chi Chang A Conditional Local Diagnosis Algorithm on the Arrangement Graph . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283 Tzu-Liang Kung, Cheng-Kuan Lin, and Yuan-Hsiang Teng Application of Artificial Intelligence Technology in the Design of Hand Training and Intelligence Training for Patients with Dementia . . . . . . . 290 Wei-Chun Hsu and Hsing-Chung Chen

xxvi

Contents

An Efficient Disaster Recovery Mechanism for Multi-region Apache Kafka Clusters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297 Lung-Pin Chen, Leu-Fang Yei, and Ying-Ru Chen Detection and Defense of DDoS Attack and Flash Events by Using Shannon Entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307 Shih-Ting Chiu, Heru Susanto, and Fang-Yie Leu 5G Base Station Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315 Yi-Cheng Jian, Meng-Shao Chung, Heru Susanto, and Fang- Yie Leu Asymmetric Cryptography Among Different 5G Core Networks . . . . . . 325 Yu-Syuan Lu, Heru Susanto, and Fang-Yie Leu The Impact of Integrating Board Games into Chinese Teaching in the Elementary School on Learning Efficiency -An Example of the Indigenous Fifth and Sixth Graders in a Remote Area of Nantou County . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334 Chiou-Shya Torng, Ho Hsiao-Yi, and Chai-Ju Lu Impacts of COVID-19 on Stock Returns of the Cross-border Transportation Industry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344 Ying-Li Lin, Kuei-Yuan Wang, and Ching-Ru Yang Study on Business Continuity of Small and Medium-Sized Firms in Japan: Focusing on Business Continuity Planning for Natural Disaster Risk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351 Mei Hua Liao and Hidekazu Sone Comparing Investor Sentiment Between Growth and Value Stocks . . . . 360 Mei-Hua Liao, Yen-Ju Chen, Chiung-Wen Yu, and Ya-Lan Chan Application of TOWS Matrix Analysis in a Precision Medicine Genetic Testing Company . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366 Kuei-Yuan Wang, Ying-Li Lin, Chien-Kuo Han, and Chia-Wei Eddie Liang Research on the Intention of the Elderly to Participate in Barrier-Free Tours . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376 Ya-Lan Chan, Wen-Qian Li, Sue-Ming Hsu, and Mei-Hua Liao Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383

A Fuzzy-Based System for Assessment of Fog Computing Resources in SDN-VANETs Ermioni Qafzezi1(B) , Kevin Bylykbashi2 , Phudit Ampririt1 , Makoto Ikeda2 , Keita Matsuo2 , and Leonard Barolli2 1

Graduate School of Engineering, Fukuoka Institute of Technology (FIT), 3-30-1 Wajiro-Higashi, Higashi-Ku, Fukuoka 811–0295, Japan {bd20101,bd21201}@bene.fit.ac.jp 2 Department of Information and Communication Engineering, Fukuoka Institute of Technology (FIT), 3-30-1 Wajiro-Higashi, Higashi-Ku, Fukuoka 811-0295, Japan [email protected], [email protected], {kt-matsuo,barolli}@fit.ac.jp

Abstract. The increase of complex data in Vehicular Ad hoc Networks (VANETs) has given rise to the vehicular cloud computing approaches. However, transferring all data to a central cloud data server is not always efficient. For time sensitive applications, it is more beneficial to distribute smaller servers closer to the premises of users. This has led to the emerging of fog and edge computing. In this paper, we propose a fuzzy-based system to assess the data processing capability of fog layer in Software Defined VANETs (SDN-VANETs). Our proposed system determines whether fog computing is appropriate and satisfies certain needs in terms of data processing. The fuzzy-based system is implemented in SDN controllers. When a vehicle needs additional resources, it can send a request to use the available resources of a fog server in its vicinity. However, for a successful data processing, the servers should meet certain requirements. The proposed system takes into consideration the time needed for sending data to the server, the load of the server and the history of previous successful tasks handled by this server. We evaluate the system by computer simulations. Fog layer adequacy is high when vehicle-to-server latency is low, server load is low and server history is very good.

1

Introduction

The long distances separating homes and workplaces/facilities/schools as well as the traffic present in these distances make people spend a significant amount of time in vehicles. Thus, it is important to offer drivers and passengers ease of driving, convenience, efficiency and safety. This has led to the emerging of Vehicular Ad hoc Networks (VANETs), where vehicles are able to communicate and share important information among them. VANETs are a relevant component of Intelligent Transportation Systems (ITS), which offer more safety and better transportation. c The Author(s), under exclusive license to Springer Nature Switzerland AG 2022  L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 1–9, 2022. https://doi.org/10.1007/978-3-031-08819-3_1

2

E. Qafzezi et al.

VANETs are capable to offer numerous services such as road safety, enhanced traffic management, as well as travel convenience and comfort. To achieve road safety, emergency messages must be transmitted in real time, which stands also for the actions that should be taken accordingly in order to avoid potential accidents. Thus, it is important for the vehicles to always have available connections to infrastructure and to other vehicles on the road. On the other hand, traffic efficiency is achieved by managing traffic dynamically according to the situation and by avoiding congested roads, whereas comfort is attained by providing in-car infotainment services. The advances in vehicle technology have made it possible for the vehicles to be equipped with various forms of smart cameras and sensors, wireless communication modules, storage and computational resources. While more and more of these smart cameras and sensors are incorporated in vehicles, massive amounts of data are generated from monitoring the on-road and on-board status. This exponential growth of generated vehicular data, together with the boost of the number of vehicles and the increasing data demands from in-vehicle users, has led to a tremendous amount of data in VANETs [10]. Moreover, applications like autonomous driving require even more storage capacity and complex computational capability. As a result, traditional VANETs face huge challenges in meeting such essential demands of the ever-increasing advancement of VANETs. The integration of cloud-fog-edge computing in VANETs is the solution to handle complex computation, provide mobility support, low latency and high bandwidth. Each of them serves different functions, but also complements eachother in order to enhance the performance of VANETs. Even though the integration of cloud, fog and edge computing in VANETs solves significant challenges, this architecture lacks mechanisms needed for resource and connectivity management because the network is controlled in a decentralized manner. The prospective solution to solve these problems is the augmentation of Software Defined Networking (SDN) in this architecture. The SDN is a promising choice in managing complex networks with minimal cost and providing optimal resource utilization. The use of an SDN Controller (SDNC) offers a global knowledge of the network with a programmable architecture which simplifies network management in such extremely complicated and dynamic environments like VANETs [9]. In addition, it will increase flexibility and programmability in the network by simplifying the development and deployment of new protocols and by bringing awareness into the system, so that it can adapt to changing conditions and requirements, i.e., emergency services [4]. This awareness allows SDN-VANET to make better decisions based on the combined information from multiple sources, not just individual perception from each node. In a previous work [8], we have proposed an intelligent approach to manage the cloud-fog-edge resources in SDN-VANETs using Fuzzy Logic (FL). We have presented a cloud-fog-edge layered architecture which is coordinated by an intelligent system that decides the appropriate resources to be used by a particular vehicle (hereafter will be referred as the vehicle) in need of additional

A Fuzzy-Based System for Assessment of Fog Computing Resources

3

Fig. 1. Logical architecture of cloud-fog-edge SDN-VANET with content distribution.

computing resources. The main objective is to achieve a better management of these resources. In another work [7], we focused only on the edge layer resources. We proposed a Fuzzy Logic System (FLS) that assessed the processing capability for each neighboring vehicle separately, hence helpful neighboring vehicles could be discovered and a better assessment of available edge resources was achieved. In this work, we propose a FLS for assessing the capability of fog layer to handle a certain application request from the vehicle, based on the server load, server history of previews accomplished tasks and vehicle-to-server latency. The remainder of the paper is as follows. In Sect. 2, we present an overview of cloud-fog-edge SDN-VANETs. In Sect. 3, we describe the proposed fuzzy-based system. In Sect. 4, we discuss the simulation results. Finally, conclusions and future work are given in Sect. 5.

2

Cloud-Fog-Edge SDN-VANETs

While cloud, fog and edge computing in VANETs offer scalable access to storage, networking and computing resources, SDN provides higher flexibility, programmability, scalability and global knowledge. In Fig. 1, we give a detailed structure of this VANET architecture. It includes the topology structure, its logical structure and the content distribution on the network. Specifically, the architecture consists of cloud computing data centers, fog servers with SDNCs, Road-Side Units (RSUs), RSU Controllers (RSUCs), Base Stations (BS) and vehicles. We also illustrate the Infrastructure-to-Infrastructure (I2I), Vehicle-toInfrastructure (V2I), and Vehicle-to-Vehicle (V2V) communication links. The

4

E. Qafzezi et al.

fog devices (such as fog servers and RSUs) are located between vehicles and the data centers of the main cloud environments. The safety applications data generated through on-board and on-road sensors are processed first in the vehicles as they require real-time processing. If more storing and computing resources are needed, the vehicle can request to use those of the other adjacent vehicles, assuming a connection can be established and maintained between them for a while. With the vehicles having created multiple virtual machines on other vehicles, the virtual machine migration must be achievable in order to provide continuity as one/some vehicle may move out of the communication range. However, to set-up virtual machines on the nearby vehicles, multiple requirements must be met and when these demands are not satisfied, the fog servers are used. Cloud servers are used as a repository for software updates, control policies and for the data that need long-term analytics and are not delay-sensitive. On the other side, SDN modules which offer flexibility and programmability, are used to simplify the network management by offering mechanisms that improve the network traffic control and coordination of resources. The implementation of this architecture promises to enable and improve the VANET applications such as road and vehicle safety services, traffic optimization, video surveillance, telematics, commercial and entertainment applications.

3

Proposed Fuzzy-Based System

In this section, we present our proposed fuzzy based system. A vehicle that needs storage and computing resources for a particular application can use those of neighboring vehicles, fog servers or cloud data centers based on the application requirements. For instance, for a temporary application that needs real-time processing, the vehicle can use the resources of adjacent vehicles if the requirements to realize such operations are fulfilled. Otherwise, it will use the resources of fog servers, which offer low latency as well. Whereas real-time applications require the usage of edge and fog layer resources, for delay tolerant applications, vehicles can use the cloud resources as these applications do not require low latency. Fog computing is a good alternative when vehicles generate massive data, which cause increased data traffic that can not be handled by the vehicles. Fog computing leverages the processing and storage capabilities of servers located in the vicinity of vehicles, RSUs, and BS. Therefore, it reduces the transmission latency and provides a solution for high throughput demands. The proposed FLS assesses the storage and processing capability of servers in fog layer. The proposed FLS is implemented in the SDNC and in the vehicles equipped with SDN modules. If a vehicle does not have an SDN module, it sends a request to the SDNC which runs the proposed system and sends back the performed evaluations. The structure of the proposed system is shown in Fig. 2. For the implementation of our system, we consider three input parameters: Current Server Load (CSL), Server History (SH), and Vehicle-to-Server Latency (VSL) to determine Fog Layer Adequacy (FLA).

A Fuzzy-Based System for Assessment of Fog Computing Resources

5

Fig. 2. Proposed system structure.

CSL: Fog servers offer higher storage and computing capability compared to vehicles, however they have their own limits. The server load indicates the number of processes waiting in queue to access the server. If the server is overloaded it might affect its performance, and even make the server unresponsive. SH: Server History gives information about previous tasks accomplished by that server. SH is defined as the ratio of the successfully accomplished tasks to the overall number of tasks performed by the server. VSL: Some applications impose strict latency requirements. For instance platooning and remote driving require communication latency of less than 25ms and 5ms, respectively [1]. The aim of fog computing is to move data collection and processing closer to where the data is produced and used. Therefore the communication latency is maintained low and a real-time communication is possible to achieve. FLA: The output parameter values consist of values between 0 and 1, with the value 0.5 working as a border to determine whether the fog layer is capable of handling the workload requested by the vehicle. We consider FL to implement the proposed system because our system parameters are not correlated with each other. Having three or more parameters which are not correlated with each other results in a non-deterministic polynomial-time hard (NP-hard) problem and FL can deal with these problems. Moreover, we want our system to make decisions in real time and fuzzy systems can give very good results in decision making and control problems [2,3,5,6,11,12]. The input parameters are fuzzified using the membership functions shown in Figs. 3(a)–3(c). In Fig. 3(d) are shown the membership functions used for the output parameter. We use triangular and trapezoidal membership functions because they are suitable for real-time operation. The term sets for each linguistic parameter are shown in Table 1. We decided the number of term sets by carrying out many simulations. In Table 2, we show the Fuzzy Rule Base (FRB) of our

6

E. Qafzezi et al. Mo

Hi

1.0 Bd

µ(SH)

µ(CSL)

1.0 Lo

0.5

0.0

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0

0.1

0.2

0.3

0.4

CSL

µ(VSL)

Md

Hg

1.0 El

µ(FLA) 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

(b)Server History

0.5

0.0

0.5

SH

(a)Current Server Load 1.0 Lw

Vg

0.5

0.0

1

Gd

0.6

0.7

0.8

0.9

1

Vl

Lw

Md

Hg

Vh

Eh

0.5

0.0

0

0.1

0.2

0.3

0.4

VSL

0.5

0.6

0.7

0.8

0.9

1

FLA

(c)Vehicle-to-Server Latency

(d)Fog Layer Adequacy

Fig. 3. Membership functions.

proposed system, which consists of 27 rules. The control rules have the form: IF “conditions” THEN “control action”. For instance, for Rule 1: “IF CLS is Lo, SH is Bd ans VSL is Lw, THEN FLA is Hg” or for Rule 20: “IF CSL is Hi, SH is Bd and VSL is Md, THEN FLA is El”. Table 1. System parameters and their term sets. Parameters

Term sets

Current Server Load (CSL)

Low (Lo), Moderate (Mo), High (Hi)

Server History (SH)

Bad (Bd), Good (Gd), Very Good (Vg)

Vehicle-to-Server Latency (VSL) Low (Lw), Moderate (Md), High (Hg) Fog Layer Adequacy (FLA)

Extremely Low (El), Very Low (Vl), Low (Lw), Moderate (Md), High (Hg), Very High (Vh), Extremely High (Eh)

4

Simulation Results

The simulations were conducted using FuzzyC and the results are shown for three scenarios. Figure 4(a), Fig. 4(b), and Fig. 4(c) show the results for low, moderate and high vehicle-to-server latency, respectively. We show the relation between FLA and CSL for different SH values.

A Fuzzy-Based System for Assessment of Fog Computing Resources

7

Table 2. FRB of FSAQoS. No CSL SH VSL FLA 1 Lo

Bd Lw

Hg

2 Lo

Bd Md

Md

3 Lo

Bd Hg

Lw

4 Lo

Gd Lw

Vh

5 Lo

Gd Md

Hg

6 Lo

Gd Hg

Md

7 Lo

Vg Lw

Eh

8 Lo

Vg Md

Vh

9 Lo

Vg Hg

Hg

10 Mo

Bd Lw

Md

11 Mo

Bd Md

Lw

12 Mo

Bd Hg

El

13 Mo

Gd Lw

Hg

14 Mo

Gd Md

Md

15 Mo

Gd Hg

Vl

16 Mo

Vg Lw

Vh

17 Mo

Vg Md

Hg

18 Mo

Vg Hg

Lw

19 Hi

Bd Lw

Vl

20 Hi

Bd Md

El

21 Hi

Bd Hg

El

22 Hi

Gd Lw

Lw

23 Hi

Gd Md

Vl

24 Hi

Gd Hg

El

25 Hi

Vg Lw

Md

26 Hi

Vg Md

Lw

27 Hi

Vg Hg

Vl

Figure 4(a) shows the scenario when a communication with a low latency from vehicle-to-server is supported. We see that due to the low latency value, the fog layer is considered suitable for processing applications, until the server load becomes high. Once the server load is overloaded, there will not be sufficient space for running other processes and applications. Even for good SH, an overloaded server will experience problems in performance and decline other access requests. The increase of VSL indicates that the fog server is located farther and therefore more time is needed for data communication. The results for moderate and high VSL are shown in Fig. 4(b) and Fig. 4(c), respectively. We can see that

8

E. Qafzezi et al. VSL = 0.1

VSL = 0.5

1 0.9

SH=0.9 SH=0.5 SH=0.1

0.9

0.8

0.8

0.7

0.7

0.6

0.6 FLA

FLA

1

SH=0.9 SH=0.5 SH=0.1

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1

0

0 0

0.1

0.2

0.3

0.4

0.5 CSL

0.6

0.7

0.8

0.9

1

0

0.1

0.2

(a)VSL = 0.1

0.3

0.4

0.5 CSL

0.6

0.7

0.8

0.9

1

(b)VSL = 0.5 VSL = 0.9

1

SH=0.9 SH=0.5 SH=0.1

0.9 0.8 0.7 FLA

0.6 0.5 0.4 0.3 0.2 0.1 0 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

CSL

(c)VSL = 0.9 Fig. 4. Simulation results.

for high VSL, the servers can be considered as suitable for helping the vehicle in need only when the server has a good SH and CSL is low.

5

Conclusions

In this paper, we proposed a fuzzy-based system to assess fog layer adequacy in a layered cloud-fog-edge architecture for SDN-VANETs. Our proposed system decides whether the fog layer is appropriate to help a vehicle that needs additional resources to accomplish different tasks. The proposed FLS is implemented in the SDNC and decides FLA based on CSL, SH, and VSL. After assessing the processing capability of fog layer, our previous proposed FS-CFELS system [8] selects the appropriate computing layer in terms of data processing. We evaluated our proposed system by computer simulations. From the simulations results, we conclude as follows. – The best FLA is achieved when communication latency between vehicle to server is low, SH is very good and Server Load is low. – In all scenarios, CSL gives the best performance when it has a low value. – The increase of VSL values deteriorates the performance of fog layer.

A Fuzzy-Based System for Assessment of Fog Computing Resources

9

In the future, we would like to make extensive simulations to evaluate the proposed system and compare the performance with other systems.

References 1. 3rd Generation Partnership Project (3GPP): Technical Specification Group Services and System Aspects; Enhancement of 3GPP Support for 5G V2X scenarios; Stage 1 (Release 16). Technical Specification 22.186 (2019). V16.2.0 2. Kandel, A.: Fuzzy Expert Systems. CRC Press Inc, Boca Raton (1992) 3. Klir, G.J., Folger, T.A.: Fuzzy Sets, Uncertainty, and Information. Prentice Hall, Upper Saddle River (1988) 4. Ku, I., Lu, Y., Gerla, M., Gomes, R.L., Ongaro, F., Cerqueira, E.: Towards software-defined VANET: architecture and services. In: 13th Annual Mediterranean Ad Hoc Networking Workshop (MED-HOC-NET), pp. 103–110 (2014) 5. McNeill, F.M., Thro, E.: Fuzzy Logic: A Practical Approach. Academic Press Professional Inc, San Diego (1994) 6. Munakata, T., Jani, Y.: Fuzzy systems: an overview. Commun. ACM 37(3), 69–77 (1994) 7. Qafzezi, E., Bylykbashi, K., Ampririt, P., Ikeda, M., Matsuo, K., Barolli, L.: A QoS-aware fuzzy-based system for assessment of edge computing resources in SDNVANETs. In: Barolli, L., Woungang, I., Enokido, T. (eds.) AINA 2021. LNNS, vol. 225, pp. 63–72. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-7510056 8. Qafzezi, E., Bylykbashi, K., Ampririt, P., Ikeda, M., Matsuo, K., Barolli, L.: An intelligent approach for cloud-fog-edge computing SDN-VANETs based on fuzzy logic: effect of different parameters on coordination and management of resources. Sensors 22(3) (2022). https://doi.org/10.3390/s22030878 9. Truong, N.B., Lee, G.M., Ghamri-Doudane, Y.: Software defined networking-based vehicular adhoc network with fog computing. In: 2015 IFIP/IEEE International Symposium on Integrated Network Management (IM), pp. 1202–1207 (2015) 10. Xu, W., et al.: Internet of vehicles in big data era. IEEE/CAA J. Automatica Sinica 5(1), 19–35 (2018) 11. Zadeh, L.A., Kacprzyk, J.: Fuzzy Logic for the Management of Uncertainty. Wiley, New York (1992) 12. Zimmermann, H.J.: Fuzzy control. In: Fuzzy Set Theory and Its Applications, pp. 203–240. Springer, Dordrecht (1996). https://doi.org/10.1007/978-94-015-87020 11

Performance Evaluation of an AI-Based Safety Driving Support System for Detecting Distracted Driving Masahiro Miwata1 , Mitsuki Tsuneyoshi1 , Makoto Ikeda2(B) , and Leonard Barolli2 1

2

Graduate School of Engineering, Fukuoka Institute of Technology, 3-30-1 Wajiro-higashi, Higashi-ku, Fukuoka 811-0295, Japan {mgm21108,mgm21106}@bene.fit.ac.jp Department of Information and Communication Engineering, Fukuoka Institute of Technology, 3-30-1 Wajiro-higashi, Higashi-ku, Fukuoka 811-0295, Japan [email protected], [email protected]

Abstract. Many accidents are caused by declining of driving skills and the lack of attention by older drivers. This is because the number of older people has been increasing. This research focuses on these issues and builds a support system for drivers. In this paper, we present the enhanced dataset of an intelligent driving support system to detect distracted driving. Our system is based on YOLO and detects multiple distracted driving behaviors by considering the driver’s hand movements. From the evaluation results, we found that the system detects distracted driving behaviors with high accuracy. Keywords: Driving support

1

· Distracted driving · YOLO

Introduction

The number of accidents caused by distracted or sleeping drivers would be reduced if more cars have better driver assistance systems [2,5,7,10,16,23]. Distracted driving has become more prevalent as a result of the development of services and In-Vehicle Infotainment (IVI) system that connect smartphones. Especially, users who have not been exposed to operating information gadgets are more susceptible to distraction while driving. In future vehicle society, some of accidents will be caused by software problems or lack of hardware maintenance. Users will need to be able to quickly update and replace the new Advanced Driver Assistance Systems (ADAS). Technology is always changing, so this will be important for reducing traffic accidents [11]. In order to realize this, devices must be low-cost and the supply of Integrated Circuits (ICs) must be reliable. Recently, the Artificial Intelligence (AI) based advanced systems have attracted attention for various fields [4,9,14,21,24]. In the automobile industry is already developed Graphics Processing Units (GPUs) for automotive applications. With AI systems focused on the edge, daily training is done in the cloud, c The Author(s), under exclusive license to Springer Nature Switzerland AG 2022  L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 10–17, 2022. https://doi.org/10.1007/978-3-031-08819-3_2

Performance Evaluation of an AI-Based Safety Driving Support System

11

and real-time detection is done at the edge [13,17,18,22]. Open datasets and models are made available [1], which allows us to accelerate the development of our apps by reducing the time required to develop applications. As a related work, a dataset of 10 different classes of distracted driving has been provided [20]. However, the dataset is outdated and does not consider vehicle equipment with ADAS or large displays for electric vehicles. In [12], we proposed an intelligent distracted driving detection system. However, the proposed system did not consider multi-function display and compare the performance with different vehicle models. In this paper, we evaluate the enhanced dataset of an AI-based driving support system to detect distracted driving. For evaluation, we discuss six cases for detecting distracted driving. The support system uses the web scrapping function to collect the images in order to improve the previous work. The structure of the paper is as follows. In Sect. 2, we describe the related work. In Sect. 3, we describe the proposed AI-based driving support system. In Sect. 4, we described the evaluation results. Finally, conclusions and future work are given in Sect. 5.

2

Related Work

Recently, many applications and detectors based on Deep Neural Networks (DNNs) have been proposed. The DNN is a network with a complex hierarchy that connects multiple internal layers for the objective of feature detection and representation learning. In the real world, representation learning is used to express the process of extracting critical information from observation data. Artificial operations are used to extract features through trial and error. However, DNN takes the image’s pixel level as an input value and learns the most appropriate characteristic to identify it [6,8]. The convolutional neural networks consider the backpropagation model like a conventional multi-layer perceptron. The authors [19] presented the effect of network depth on image classification accuracy in large-scale configurations. The YOLO algorithm [15] has been proposed by Joseph Redmon and et al. He stopped developing due to military use and privacy issues. Then, Alexey Bochkovskiy has taken over the development and released YOLOv4 in April 2020 [3]. Glenn Jocher and et al. released YOLOv5 in June 2020, which utilizes PyTorch as the machine learning library. YOLO series are end-to-end form by one-stage object detection network. As a result, the detector is more rapid than two stage detector. The YOLO algorithm takes the whole image as input, divides the image into a grid and predicts the whole image directly. This effectively avoids background errors by making use of the environmental information available.

12

M. Miwata et al.

3

AI-Based Driving Support System

3.1

Overview of the Driver Support System

We show the graphical structure of the proposed AI-based driving support system in Fig. 1. The proposed system is comprised of an intelligent driving assistance system that is used in the vehicle. The AI-based system is an application of the proposed system that predicts the object and alerts the driver. The edge is integrated into the Jetson Xavier NX and connects to the Internet in order to obtain training images through the use of the web scraping function. YOLOv5 is used for computing the object detection.

Fig. 1. Flow of the proposed system.

3.2

Distraction Behaviors for Driving

It is not always possible for both hands to be on the steering wheel at the same time while driving, and their positions change throughout the trip. Enhanced In-Vehicle Infotainment (IVI) is now standard on many modern vehicles, and features include navigation, geo-location, voice communication, Internet access, search functions for news and e-mail, and more. A lot of information is available to drivers, enabling them to drive safely. However, the drivers sometimes could be expected to make the following actions. • • • • • • • • •

Use only one hand to operate the IVI. Use your voice to control the IVI. Taking a look at the augmented reality navigation. Use a cellular phone to communicate. Looking aside. Having a conversation with a passenger. Holding a bottle in order to consume a soft drink or some water. Using your fingers to rub your eyes or to touch your glasses or sunglasses. Fall asleep or lose consciousness.

We show examples of miss detection when we do not use the original dataset in Fig. 2(a) and Fig. 2(b). In these examples, multi-function display and window switches are miss detected as smartphone. Figure 2(c) and Fig. 2(d) show the non-detected cases when our previous dataset [12] is used. In these examples, the detection system was unable to detect the multi-function displays. This paper investigates how to detect distracted driving as well as how to reduce the number of false positives.

Performance Evaluation of an AI-Based Safety Driving Support System

(a) Miss detected: case #1

(b) Miss detected: case #2

(c) Non-detected: case #1

(d) Non-detected: case #2

13

Fig. 2. Examples that have been missed or have not been detected by previous work.

4 4.1

Evaluation Results Overview of Dataset and Settings

In this work, we took additional photos in several vehicles to collect the normal and distracted driving images for enhancing an original dataset. The dataset contains 1, 200 images. The YOLOv5m-P5 model serves as the basis for our network model. We utilized a maximum of 20 epochs to generate a training model for this evaluation. The metrics such as recall, precision and mean Average Precision (mAP) are used in the evaluation. Recall indicates the fraction of relevant instances that were retrieved. Precision indicates the fraction of relevant instances among the retrieved instances. The mAP indicates the mean of Average Precision (AP) in all classes. The mean accuracy of a class is usually denoted by AP. 4.2

Results

Figure 3 shows the results of distracted driving detection in several driver’s situations. The detection results for different six cases are described below.

14

M. Miwata et al.

(a) Case #A

(b) Case #B

(c) Case #C

(d) Case #D

(e) Case #E

(f) Case #F

Fig. 3. Detection results for different six cases.

• Case #A: Detection of multi-display (monitor) and one hand drive and control of the smartphone on the lap with your left hand, • Case #B: Detection of monitor and control of the smartphone with your right hand, • Case #C: The same as in Case A, but with a different model of vehicle and clothes, • Case #D: Detection of monitor and one hand drive, control of the smartphone placed in the console box with your left hand, • Case #E: Detection of monitor and two hands drive, • Case #F: Detection of left hand operation of the monitor and detection of one hand drive with your right hand. From these results, we found that our model was able to detect distracted driving behaviors with high accuracy. In this case, our system outputs the smartphone

Performance Evaluation of an AI-Based Safety Driving Support System

15

as a tag of classes and it appears in the bounding box (see Fig. 3(a) to Fig. 3(d)). The detection algorithm provides advantage for the overlap between smartphone and one hand. The monitor is detected in all cases, even though the whole form is not shown in Fig. 3(b). When the smartphone and hand are isolated, our system does not detect distracted driving (see Fig. 3(e) and Fig. 3(f)). In addition, Fig. 3(f) shows the motion of distracted driving if the monitor and one-handed bounding boxes are overlapped. The results of each evaluation metrics are shown in Fig. 4. We observed that each parameter shows high accuracy and becomes stable after only 10 epochs. The results of recall and mAP reached the highest score in this evaluation.

(a) Recall

(b) Precision

(c) mAP

Fig. 4. Evaluation results.

5

Conclusions

In this paper, we evaluated the enhanced dataset of intelligent driving support system to detect the distracted driving. We confirmed that the accuracy in the seven cases was high due to the enhanced dataset and improving the previous work. In the future work, we will consider more motions to improve the reliability of our proposed dataset.

References 1. Kaggle: Data science community. https://www.kaggle.com/ 2. Bergasa, L.M., Almeria, D., Almazan, J., Yebes, J.J., Arroyo, R.: DriveSafe: an app for alerting inattentive drivers and scoring driving behaviors. In: Proceedings of the IEEE Intelligent Vehicles Symposium 2014, pp. 240–245 (2014)

16

M. Miwata et al.

3. Bochkovskiy, A., Wang, C.Y., Liao, H.Y.M.: YOLOv4: optimal speed and accuracy of object detection. Computer Vision and Pattern Recognition (cs.CV), April 2020. https://arxiv.org/abs/2004.10934 4. Chen, G., et al.: NeuroIV: neuromorphic vision meets intelligent vehicle towards safe driving with a new database and baseline evaluations. IEEE Trans. Intell. Transp. Syst. 23(2), 1171–1183 (2022) 5. Ersal, T., Fuller, H.J.A., Tsimhoni, O., Stein, J.L., Fathy, H.K.: Model-based analysis and classification of driver distraction under secondary tasks. IEEE Trans. Intell. Transp. Syst. 11(3), 692–701 (2010) 6. Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006) 7. Kandeel, A.A., Elbery, A.A., Abbas, H.M., Hassanein, H.S.: Driver distraction impact on road safety: a data-driven simulation approach. In: Proceedings of the IEEE Global Communications Conference (GLOBECOM 2021), pp. 1–6, December 2021 8. Le, Q.V.: Building high-level features using large scale unsupervised learning. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing 2013 (ICASSP 2013), pp. 8595–8598, May 2013 9. Li, B., et al.: A new unsupervised deep learning algorithm for fine-grained detection of driver distraction. IEEE Trans. Intell. Transp. Syst. 1–13 (2022) 10. Liu, T., Yang, Y., Huang, G.B., Yeo, Y.K., Lin, Z.: Driver distraction detection using semi-supervised machine learning. IEEE Trans. Intell. Transp. Syst. 17(4), 1108–1120 (2016) 11. McCall, J.C., Trivedi, M.M.: Driver behavior and situation aware brake assistance for intelligent vehicles. Proc. IEEE 95(2), 374–387 (2007) 12. Miwata, M., Tsuneyoshi, M., Tada, Y., Ikeda, M., Barolli, L.: Design of an intelligent driving support system for detecting distracted driving. In: Proceedings of the 15th International Conference on Complex, Intelligent, and Software Intensive Systems (CISIS 2021), pp. 377–382, July 2021 13. Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015) 14. Poon, Y.S., Lin, C.C., Liu, Y.H., Fan, C.P.: YOLO-based deep learning design for in-cabin monitoring system with fisheye-lens camera. In: Proceedings of the IEEE International Conference on Consumer Electronics (ICCE 2022), pp. 1–4, January 2022 15. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2016), pp. 779–788, June 2016 16. Shaout, A., Roytburd, B., Sanchez-Perez, L.A.: An embedded deep learning computer vision method for driver distraction detection. In: Proceedings of the 22nd International Arab Conference on Information Technology (ACIT 2021), pp. 1–7, December 2021 17. Silver, D., et al.: Mastering the game of Go with deep neural networks and tree search. Nature 529, 484–489 (2016) 18. Silver, D., et al.: Mastering the game of Go without human knowledge. Nature 550, 354–359 (2017) 19. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Proceedings of the 3rd International Conference on Learning Representations (ICLR 2015), May 2015 20. State Farm: Dataset of state farm distracted driver detection (2016). https://www. kaggle.com/c/state-farm-distracted-driver-detection/

Performance Evaluation of an AI-Based Safety Driving Support System

17

21. Ugli, I.K.K., Hussain, A., Kim, B.S., Aich, S., Kim, H.C.: A transfer learning approach for identification of distracted driving. In: Proceedings of the 24th International Conference on Advanced Communication Technology (ICACT 2022), pp. 420–423, February 2022 22. Vicente, F., Huang, Z., Xiong, X., la Torre, F.D., Zhang, W., Levi, D.: Driver gaze tracking and eyes off the road detection system. IEEE Trans. Intell. Transp. Syst. 16(4), 2014–2027 (2015) 23. Wang, Y.K., Jung, T.P., Lin, C.T.: EEG-based attention tracking during distracted driving. IEEE Trans. Neural Syst. Rehabil. Eng. 23(6), 1085–1094 (2015) 24. Xing, Y., Lv, C., Wang, H., Cao, D., Velenis, E., Wang, F.Y.: Driver activity recognition for intelligent vehicles: a deep learning approach. IEEE Trans. Veh. Technol. 68(6), 5379–5390 (2019)

Autonomous Migration of Virtual Machines to Reduce Energy Consumption of Servers Dilawaer Duolikun1(B) , Tomoya Enokido2 , Leonard Barolli3 , and Makoto Takizawa1 1

2

RCCMS, Hosei University, Tokyo, Japan [email protected], [email protected] Faculty of Business Administration, Rissho University, Tokyo, Japan [email protected] 3 Department of Information and Communications Engineering, Fukuoka Institute of Technology, Fukuoka, Japan [email protected]

Abstract. It is critical to reduce the electric energy consumption of information systems to realize green societies. In this paper, we take the live virtual machine migration approach to reducing the energy consumption of servers. In our previous studies, each server is assumed to be able to obtain the local state of every other server like the number of active processes and anytime accept any virtual machine from another server. In reality, each server cannot obtain the local state of another server without communicating with each other. In addition, a server cannot accept a virtual machine from another server if the server is overloaded. In this paper, we newly propose an AMVM (Asynchronous Migration of Virtual Machines) protocol to synchronize servers to decide on which server to make a virtual machine migrate to which server. Keywords: Autonomous migration of virtual machines computing systems · AMVM protocol

1

· Green

Introduction

In scalable information systems like the IoT (Internet of Things), the electric energy consumption of clouds of servers [2–5,13] has to be reduced to decrease the carbon dioxide emission [23]. In this paper, we consider the live migration approach of virtual machines [1,13] to reducing the energy consumption of servers in clouds [1–5,13]. Energy-aware algorithms [6,9–11,20–22] are proposed to select a host server and a virtual machine on the host server to perform an application process. In addition, virtual machines migrate from host servers to guest servers to reduce the energy consumption of the servers in the migration approach [6,13–16,21]. Systems can be made more reliable by replicating processes on virtual machines [7,8]. c The Author(s), under exclusive license to Springer Nature Switzerland AG 2022  L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 18–30, 2022. https://doi.org/10.1007/978-3-031-08819-3_3

Autonomous Migration of Virtual Machines

19

Power consumption and computation models of servers [2–5,11] and fog nodes [25,26] are proposed. By using the models, the execution time and energy consumption of a server to perform processes are obtained by simulating the computation steps of the processes as discussed in the SM (SiMulation) algorithm [9–11]. However, it is difficult to a priori obtain the computation residue of each active process and it takes time to do the simulation. In our previous studies on energy-efficient virtual machine migration algorithms [9–21], each server is assumed to be able to obtain the current state of another server, e.g. the number of active processes and anytime accept any virtual machine from another server. In reality, each server cannot obtain the local state of another server without communicating with each other. In addition, a server cannot accept a virtual machine from another server if the server is overloaded. Each server has to communicate with other servers to decide on which virtual machine migrates to which server. In this paper, we newly propose an AMVM (Asynchronous Migration of Virtual Machines) protocol by which each server communicates with another server to get the local state and the migration permission of virtual machines. In Sect. 2, we present the computation and power consumption models of a server. In Sect. 3, we propose the energy estimation algorithm. In Sect. 4, we discuss the centralized and distributed migration algorithms. In Sect. 5, we propose the AMVM protocol.

2

System Model

A cloud C is composed of servers s1 , . . . , sm (m ≥ 1). Each server st is equipped with npt (≥ 1) homogeneous CPUs, each of which supports cnt (≥ 1) cores and each core supports tnt (≥ 1) threads. The server st totally supports nct (= cnt ·pct ) cores and ntt (= nct ·tnt ) threads. In this paper, a process stands for an application process which uses CPU resources [3]. A process being performed is active. A server is active if at least one process is active, otherwise idle. Time is modeled to be a discrete sequence of time units [tu]. SP t (τ ) is a set of active processes on a server st at time τ . Processes issued by clients are performed on a virtual machine vmk [1] resident on a server st independently of the location and heterogeneity of each server. V P k (τ ) (⊆ SP t (τ )) is a set of active processes on a virtual machine vmk at time τ . A virtual machine vmk is active iff |V P k (τ )| > 0, otherwise idle. A virtual machine vmk is smaller than or equivalent with vmh (vmk ≤ vmh ) iff |V P k (τ )| ≤ |V P h (τ )|. A virtual machine vmk on a host server sh can migrate to a guest server sg in a live manner [1]. Here, the migration time tm of a virtual machine is about two [sec] according to the experiment [21]. Active processes on vmk do not terminate but are just suspended during the migration time tm. V M t is a set of virtual machines on a server st . The execution time of a process pi on a server st is the same as a virtual machine vmk resident on the server st as discussed in paper [24].

20

D. Duolikun et al.

The power consumption N Et (nt ) [W] of a server st to perform nt (= |SP t (τ )|) active processes is given in the MLPCM (Multi-Level Power Consumption) model [10,11,13]. Each time a CPU, core, and thread are activated, the power consumption of a server st increases by bE t , cE t , and tE t [W], respectively. ⎧ minEt if nt = 0. ⎪ ⎪ ⎪ ⎪ + n ·(bE + cE + tE ) if 1 ≤ nt ≤ npt . minE ⎪ t t t t t ⎪ ⎨ if npt < nt ≤ nct . minEt + npt ·bEt + nt ·(cEt + tEt ) N E t (nt ) = (1) if nct < nt < ntt . minEt + npt ·bEt + nct ·cEt + nt ·tEt ⎪ ⎪ ⎪ ⎪ maxEt (= minEt + npt ·bEt + nct ·cEt + ntt ·tE t ) ⎪ ⎪ ⎩ if nt ≥ ntt . The power Et (τ ) [W] consumed by a server st to perform nt (= |SP t (τ )|) processes at time τ is defined to be N E t (nt ) assuming CPUs, cores, and threads are fairly allocated to processes. Energy consumed by a server st from time st et [tu] to time et is τ =st N Et (|SPt (τ )|) [W tu]. The execution time of each process depends on how many processes are active on a thread. minT i is the minimum execution time [tu] of a process pi on a server st where only the process pi is active on a thread without any other process. Let minT i be a minimum one of minT 1i , . . ., minT mi of the servers s1 , . . . , sm in the cloud C. The amount of computation of each process pi is defined to be minT i [2–5]. The computation residue RP i of each process pi is minT i when the process pi starts. It is noted minT i /minT ti = minT j /minT tj = T CRt (≤ 1) for any pair of processes pi and pj on a server st . Here, T CRt is the thread computation rate of a server st . If T CRt > T CRu , a server st is faster than another server su . If only one process pi is active on a thread of a server st , the process pi is performed at rate T CRt . This means, the computation residue RP i is decremented by T CRt for one time unit. If l (> 0) processes are active on a thread, each of the processes is performed at rate T CRt /l. On a server st with nt active processes, each process is performed at rate N P Rt (nt ) in the MLC (Multi-Level Computation) model [10,11] where N P Rt (nt ) = T CRt for 0 < nt ≤ ntt , T CRt · (ntt /nt ) for nt > ntt . The server computation rate 0. N SRt (nt ) (≤ ntt ·T CRt ) of a server st is N P Rt (nt )·nt for nt > The total computation residue RS t of a server st at time τ is pi ∈SP t (τ ) RP i . The computation residue RS t is decremented by N SRt (|SP t (τ )|) at each time. We define the power-computation (P C) rate P CRt (nt ) of a server st to be N E t (nt )/N SRt (nt ) (nt > 0). One unit of computation is defined to be the computation which takes one [tu] on the fastest server st where T CRt = 1. P CRt (nt ) shows the power consumption of a server st to perform one computation unit, where nt processes are active. If nt ≥ ntt , P CRt (nt ) is a maximum value P CRt (ntt ) = maxE t /(ntt · T CRt ). We present a computation model of processes on a server st . Variables Ct , Tt , and Et denote a set of active processes, active time and energy consumption of each server st , respectively. RP i and Ti show the computation residue and execution time of each process pi , respectively. At each time τ , if a process pi

Autonomous Migration of Virtual Machines

21

starts on a server st , RP i is minT i . Ti is incremented by one if RP i > 0. Et is incremented by N E t (|Ct |). RP i of each process pi in the set Ct is decremented by N P Rt (nt ). Then, if RP i ≤ 0, pi terminates and is removed from the set Ct . [Computation model of processes on a server st ] 1. Initially, Et = 0; Ct = φ; Tt = 0; τ = 1; 2. while () (a) for each process pi which starts on a server st at time τ , Ct = Ct ∪ {pi }; RP i = minT i ; Ti = 0; (b) nt = |Ct |; Et = Et + N E t (nt ); if nt > 0, Tt = Tt + 1; (c) for each process pi in Ct , Ti = Ti + 1; RP i = RP i − N P Rt (nt ); if RP i ≤ 0, Ct = Ct − {pi }; (d) τ = τ + 1; In well-formed applications like transaction systems, processes are daily used and it is easy to get minT i of each process pi in a cluster.

3

Estimation Algorithms of Energy Consumption

We discuss how to estimate electric energy to be consumed by a server to perform processes. In papers [17–20], the estimation algorithms like the MI [17] and SMI [20] are proposed where RP i ≤ RP j or RP i ≥ RP j for every pair of active processes pi and pj on a server. This means, the number of active processes monotonically decreases as time advances. Under the assumption that no process is to be newly issued to a server st after current time τ , the energy consumption of the server st can be more precisely estimated [17]. On the other hand, the estimation algorithms are more complex, i.e. it takes longer time to perform the estimation algorithms. In this paper, we make a more practical assumption that the minimum execution time minT i of each process pi is a priori known, i.e. every process is well defined like a transaction daily used. In addition, we consider the total computation residue RS t of all the active processes on each server st at a macro level and do not consider the computation residue of each active process. Let nt be the number of active processes and RS t be the total computation residue of active processes on a server st at time τ . Initially, each server st is idle, i.e. nt = 0 and RS t = 0. If a process pi is issued to a server st , the total computation residue RS t is incremented by minT i . At each time unit, RS t is decremented by the server computation rate N SRt (nt ). Suppose nt processes are active on a server st at time τ whose total computation residue is RS t . The execution time ET t (RS t , nt ) of the nt active processes is defined to be RS t /N SRt (nt ) [tu]. That is, RS t > 0 and RS t = 0 before and after time τ + ET t (RS t , nt ), respectively. The server st consumes the energy EC t (RS t , nt ) = N E t (nt ) [W] · ET t (RS t , nt ) [tu] = RS t /N SRt (nt ) · N E t (nt ) = RS t · P CRt (nt ) [W tu] to perform the nt active processes.

22

D. Duolikun et al.

First, we consider the energy consumption HGEC hg (= EC h (RS h , nh ) + EC g (RS g , ng )) of the servers sh and sg among which no virtual machine migrates. (2) HGEC hg = RS h · P CRh (nh ) + RS g · P CRg (ng ). Next, the virtual machine vmk migrates from the host server sh to the guest server sg . Processes on the virtual machine vmk are suspended at time τ to time τ + tm and restarts on the guest server sg at time τ + tm where tm is the migration time. Since the nv k processes leave the server sh , the total computation residue RV k of the virtual machine vmk is considered to be RS h · nv k /nh and the computation residue RS h of the host server sh is reduced to RS h − RV k = RS h · (1 − nv k /nh ). The number nh of active processes on the host server sh is also reduced to nh − nv k . The execution time of the server sh is RS h · (1 − nv k /nh )/N SRh (nh −nv k ) [tu]. The host server sh consumes the energy HEC h:k = EC h (RS h −RV k , nh −nv k ) = RS h ·(1−nv k /nh )·N E h (nh −nv k )/N SRh (nh − nv k ) = RS h · (1 − nv k /nt ) · P CRh (nh − nv k ) [W tu]. For tm [tu] from time τ , processes in the set SP g (τ ) are performed on the guest server sg while active processes on the virtual machine vmk are suspended. Here, the computation N SRg (ng ) · tm is performed, i.e. the computation residue RS g of the guest server sg is reduced to RS g − N SRg (ng ) · tm if tm < RS g /N SRg (ng ). This means, some process in the set SP g (τ ) is still active at time τ + tm. The guest server sg consumes the energy N E g (ng ) · tm since the power N E g (ng ) [W] is consumed for tm [tu]. The computation residue RS g is reduced to RS g − N SRg (ng ) · tm. At time τ + tm, nv k processes on the virtual machine vmk newly restart in addition to the ng processes on the guest server sg . Hence, the computation residue RS g increases by the computation residue RV k , i.e. RS g increases to [RS g − N SRg (ng )·tm]+RS h ·(nv k /nh ). The number of active processes also increases by the number nv k of active processes on the virtual machine vmk , i.e. ng + nv k . Hence, the execution time of of the guest server sg is tm + [RS g − N SRg (ng ) · tm + RS h · (nv k /nh )]/N SRg (ng + nv k ) [tu]. The guest server sg consumes the energy GEC g:k = N E g (ng ) · tm + [RS g − N SRg (ng ) · tm + RS h · (nv k /nh )] · P CRg (ng + nv k ) [W tu]. Next, we consider case no process is active at time τ + tm when the virtual machine vmk restarts on the guest server sg . That is, all the ng processes terminate at time τ + RS g /N SRg (ng ) (< τ + tm) and then no process is active until time τ + tm when the virtual machine vmk restarts on the server sg . Then, the nv k processes on the virtual machine vmk are performed on the guest server sg at time τ + tm. It takes ET g (RV k , nv k ) = RS h · (nv k /nh )/N SRg (nv k ) [tu] to perform the nv k processes and the guest server sg consumes the energy EC g (RV k , nv k ) = RS h · (nv k /nh )/N SRg (nv k ) · N E g (nv k ) = RS h · (nv k /nh ) · P CRg (nv k ) [W tu]. Thus, the total execution time to perform the ng processes and the nv k processes on the virtual machine vmk is tm + RS h · (nv k /nh )/N SRg (nv k ) [tu]. The server sg consumes the minimum power minE g [W] from time τ + RS g /N SRg (ng ) to τ + tm. Hence, the guest server sg consumes the energy GEC g:k = RS g · P CRg (ng ) + minE g · (tm −

Autonomous Migration of Virtual Machines

RS g /N SRg (ng )) + RS h · (nv k /nh ) · P CRg (nv k ) [W tu]. ⎧ N E g (ng ) · tm + [RS g − N SRg (ng ) · tm + RS h · (nv k /nh )]· ⎪ ⎪ ⎨ if tm ≤ RS g /N SRg (ng ). P CRg (ng + nv k ) GEC g:k = · P CR (n ) + minE RS ⎪ g g g g · (tm − RS g /N SRg (ng ))+ ⎪ ⎩ RS h · (nv k /nh ) · P CRg (nv k ) otherwise.

23

(3)

Thus, the servers sh and sg totally consume the energy M EC hg:k = HEC h:k + GEC g:k .

4

Centralized and Distributed Algorithms

Clients issue processes to a cloud and the processes are hen performed on virtual machines. In addition, virtual machines migrate from host servers to guest servers. Here, host servers and guest servers have to be selected to reduce the total energy consumption of the servers. Algorithms to select host servers to perform processes and to make virtual machines migrate to guest servers are so far proposed [9–11,13,15,16]. In these algorithms, a centralized controller is assumed to exist, which can obtain the current state of each server, i.e. number nt of active processes and total computation residue RS t of each server st . By using the state information of the servers, the controller decides on which virtual machine on which server to migrate to which guest server to reduce the total energy consumption of the servers. For example, in the VM (Virtual machine Migration) algorithm [20], the controller periodically checks the migration condition for every server and makes virtual machines migrate to guest servers as follows: [VM algorithm] 1. S = a set of all the servers; AS = a set of active servers; τ = current time; 2. for every server st in S, nt = number of active processes on st ; RS t = computation residue of sh ; V M t = set of virtual mchines on st ; 3. while (AS = φ) do (a) Select an active server sh in AS whose energy EC h (RS h , nh ) is the largest; S = S − {sh }; /* sh is not selected as a guest server */ (b) Select a smallest active virtual machine vmk in V M h ; (c) Select a server sg in S where HGEC hg > M EC hg:k and M EC hg:k is the smallest; (d) If sg is found, the virtual machine vmk migrates from sh to sg ; V M h = V M h − {vmk }; V M g = V M g ∪ {vmk }; If sg in AS, AS = AS − {sg }; /* sg is not selected as a host server */ If there is no active virtual machine on sh , AS = AS − {sh }; (e) If sg is not found, AS = AS − {sh };

24

D. Duolikun et al.

It is not easy to realize the centralized controller, especially in scalable clouds of servers due to the heavy communication and computation overhead. We consider a distributed algorithm where each server sh autonomously checks the migration condition. Only if the migration condition is satisfied at current time τ , the server sh selects a guest server sg and makes a virtual machine vmk migrate to the guest server sg . [Distributed VM (DVM) algorithm] 1. Initially, lastmgth = a random value from −mgint/2 to mgint/2; 2. At each time τ , if lastmgth + mgint ≥ τ and the migration condition, i.e. nt /2 > ntt is satisfied, each active server st selects a smallest active virtual machine vmk resident on the server sh ; 3. The server st selects a guest server sg in the cloud where HGEC hg > M EC hg:k and M EC hg:k is the smallest; 4. The virtual machine vmk migrates to the guest server sg ; lastmgth = τ ; Here, lastmgth stands for most recent time some virtual machine migrates from a host server sh . At time τ = lastmgth + mgint, each active server sh checks the migration condition “nt /2 > ntt ”. That is, each active server sh checks the migration condition mgint time units later after a virtual machine migrates. If the migration condition is satisfied, the server sh selects a smallest active virtual machine vmk . Then, the server sh selects a guest server sg where the total energy consumption of the servers sh and sg can be reduced, i.e. HGEC hg > M EC hg:k , and the energy consumption M EC hg:k is smallest, where the virtual machine vmk migrates. In order to evaluate the VM and DVM algorithms, we consider a cloud of eight servers s1 , . . . , s8 (m = 8) used in paper [20] whose energy and performance parameters are shown in Table 1. Each server initially supports twenty idle virtual machines and there are totally 160 virtual machines in the cloud. Totally n processes p1 , . . . , pn are randomly issued from time 1 to xtime, where xtime is 1,000 [tu]. For each process pi , the starting time stimei is randomly taken from 1 to xtime [tu]. The minimum execution time minT i of each process pi is randomly taken from 1 to 20 [tu] so that the average value of minT 1 , . . . , minT n is 10 [tu]. A process configuration P F (n) of n processes p1 , . . . , pn is a set {i, minT i , stimei  | i = 1, . . . , n} of n tuples. Each tuple i, minT i , stimei  denotes a process pi . Given the number n of processes, twenty process configurations are randomly generated. Then, for each process configuration P F (n), the VM and DVM algorithms are executed according to the computation model presented in Sect. 2. Then, the total energy consumption T E [W tu] of the servers are obtained. The average value of T Es obtained for twenty process configurations P F (n) is calculated for each number n of processes. The migration check interval mgint is four [tu]. The migration time tm is one [tu]. As presented in paper [21], the migration time tm is two [sec]. Hence, one time unit is considered to be 0.5 [sec] in the evaluation.

Autonomous Migration of Virtual Machines

25

Table 1. Parameters of servers sid T CR nb nc nt minE[W] bE[W] cE[W] tE[W] maxE[W] P CR 1,5 1

1

8

16 250

25

12

4

435

27.2

2,6 0.9

1

8

16 200

20

10

2

332

23.1

3,7 0.7

1

4

8 180

15

8

1

235

42.0

4,8 0.5

1

2

4 100

10

6

1

126

63.0

Total energy consumption of servers [k W tu]

Figure 1 shows the total energy consumption T E [W tu] of the servers in the centralized VM and distributed DVM algorithms. As shown in Fig. 1, the servers consume more amount of energy in the DVM algorithm than the VM algorithm. Especially, the energy consumption of the servers drastically increases in the DVM algorithm for n > 10, 000. Since each server makes a migration decision independently of other servers, a pair of servers may send virtual machines to each other at the same time. If one of the servers sends a virtual machine to the other server, the total energy consumption of the servers can be reduced. Thus, we need some synchronization mechanism on which server to make a virtual machine migrate to which server in a cloud. 18000

VM 16000 DVM 14000 12000 10000 8000 6000 4000 2000 2

4

6

8

10

12

14

16

18

Number n (k) of processes

Fig. 1. Total energy consumption T E of servers.

5

An AMVM (Autonomous Migration of Virtual Machines) Protocol

We newly propose an AMVM (Autonomous Migration of Virtual Machines) protocol to synchronize servers in a cloud to decide on which host server to make a virtual machine to a guest server. Each server sh autonomously checks if an active virtual machine should migrate to a guest server sg . In order to

26

D. Duolikun et al.

make the migration decision, each server sh has to obtain the local state of another server sg , i.e. the number ng of active processes and the computation residue RS g . In addition, even if a virtual machine vmk and a guest server sg are decided by a host server sh , vmk cannot migrate to sg without getting the migration permission from sg . A server which would like to make a virtual machine migrate to another server is referred to as a requesting server. On the other hand, a server which is asked to receive a virtual machine from another server is a passive server. A server which is neither requesting nor passive is a neutral server. In this paper, each server can be only one of requesting, passive, and neutral ones. Each neural active server st checks if a virtual machine should migrate to a guest server. In this paper, if the following MG (MiGration) condition is satisfied at current time τ , an active server st plays a requesting role and selects an active virtual machine and a guest server. [MG condition] 1. τ ≥ lastmgth + mgint. 2. A server st is active and neutral. 3. nt /ntt > 2. In the condition 1, lastmgth shows most recent time a virtual machine migrates from a server sh as presented in the DVM algorithm. The condition 2 means that there is at least one active virtual machine on the server st . The condition 3 means that at least two processes are active on each thread of a server st . As discussed in Sect. 2, if nt ≤ ntt , even if some process leaves a host server sh , the execution time of each active process pi does not change since only the process pi is active on a thread. Each neutral active server sh checks the MG condition at each time τ . If the MG condition is satisfied, the server sh transits to requesting state. Even if a requesting server sh is asked to be a passive server from another requesting server, the server sh just rejects the request. Thus, each server can be either a requesting or passive server. A requesting server sh behaves as follows: [Requesting server] 1. If the MG condition is satisfied for an active server sh , the server sh transits to requesting state and selects a smallest active virtual machine vmk resident on the server sh ; nv k = number of active processes on the virtual machine vmk ; RV k = computation residue RS h · nv k /nh of the virtual machine vmk ; 2. HE h = CE h (RS h , nh ) = RS h · P CRh (nh ); V HE h = HEC h:k ; 3. The server sh sends an MGQ (Migration reQuest) message RS h , nh , RV k , nv k , HE h , V HE h  to servers in the cloud. 4. The server sh waits for an MGR (MG reply) message RS g , ng , GE g , V GE g  from each server sg to which the server sh sends the M GQ message;

Autonomous Migration of Virtual Machines

27

5. The server sh collects MGR messages from servers in the cluster; GS h = a set of servers from which the server sh receives MRQ messages; 6. while (GS h = φ) do (a) The server sh selects a server sg in the set GS h where the energy consumption V HE h + V GE g is the smallest; (b) The server sh sends an M G (MiGrate) message to the server sg ; (c) If the server sh receives an ACK message from the server sg , the virtual machine vmk migrates to the server sg ; SGh = Gh − {sg }; (d) Otherwise, i.e. the server sh receives a N AK message, SGh = Gh − {sg }; At the step 2, the server sh calculates on the energy consumption HE h to perform the nh processes, where no virtual machine migrates from the server sh . The server sh also obtains the energy consumption V HE h to perform (nh −nv k ) processes, where the virtual machine vmk migrates from the server sh . A neutral server sg transits to passive state if the server sg receives an MGQ message from a requesting server sh . A passive server sg behaves as follows: [Guest server] 1. On receipt of an MGQ message RS h , nh , RS k , nv k , HE h , V HE h  from a server sh , if sg is not neutral, sg sends a N O message to the host server sh ; Otherwise, the neutral server sg becomes a passive server; GE g = CE g (RS g , ng ) = RS g /P CRg (ng ); V GE g = GEC g:k ; 2. If HE h + GE g > V HE h + V GE g , the server sg sends an MGR (MG Reply) message RS g , ng , GE g , V GE g  to the server sh and transits to passive state; Otherwise, the server sg sends a N O message to the server sh and backs to neutral state. 3. On receipt of the M G message from the server sh , the server sg sends an ACK message to sh if sg can accept the virtual machine vmk . 4. Otherwise, the server sg sends a N AK message to the server sh and transits to neutral state; 5. On receipt of the virtual machine vmk , processes on the virtual machine vmk restart on the server sg if the server sg is passive; At step 1, the server sg calculates on the energy consumption GE g to perform the ng processes, where no virtual machine migrates to the server sg . The server sg also calculates on the energy consumption V GE g to perform not only ng processes but also nv k processes on the virtual machine mv k . At step 2, only if the total energy consumption of the servers sh and sg can be reduced by making vmk migrate to sg , sg sends the M GR message to sh . At step 3, the server sg decides on whether or not sg accept vmk .

6

Concluding Remarks

It is critical to reduce the energy consumption of servers to realize green societies. In scalable information systems like cloud computing systems, it is not easy to

28

D. Duolikun et al.

decide on which virtual machine on which host server to migrate to which guest server in a centralized way. In this paper, we newly discuss the distributed way, the autonomous migration of virtual machines, where each server autonomously selects a guest server to which a virtual machine migrates and accepts a virtual machine from another server. Here, each server autonomously decides on which virtual machine to migrate to which guest server. We proposed the AAVM protocol to do the negotiation among servers to decide on a pair of a host server and a guest server to make a virtual machine migrate. We are now evaluating the AAVM protocol.

References 1. KVM: Main Page - KVM (Kernel Based Virtual Machine) (2015). http://www. linux-kvm.org/page/Mainx Page 2. Enokido, T., Aikebaier, A., Takizawa, M.: Process allocation algorithms for saving power consumption in peer-to-peer systems. IEEE Trans. Ind. Electron. 58(6), 2097–2105 (2011) 3. Enokido, T., Aikebaier, A., Takizawa, M.: A model for reducing power consumption in peer-to-peer systems. IEEE Syst. J. 4(2), 221–229 (2010) 4. Enokido, T., Aikebaier, A., Takizawa, M.: An extended simple power consumption model for selecting a server to perform computation type processes in digital ecosystems. IEEE Trans. Ind. Inform. 10(2), 1627–1636 (2014) 5. Enokido, T., Takizawa, M.: Integrated power consumption model for distributed systems. IEEE Trans. Ind. Electron. 60(2), 824–836 (2013) 6. Enokido, T., Duolikun, D., Takizawa, M.: The energy consumption laxity-based algorithm to perform computation processes in virtual machine environments. Int. J. Grid Util. Comput. 10(5), 545–555 (2019) 7. Enokido, T., Duolikun, D., Takizawa, M.: The improved redundant active timebased (IRATB) algorithm for process replication. In: Barolli, L., Woungang, I., Enokido, T. (eds.) AINA 2021. LNNS, vol. 225, pp. 172–180. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-75100-5 16 8. Enokido, T., Duolikun, D., Takizawa, M.: The improved redundant active timebased algorithm with forcing termination of meaningless replicas in virtual machine environments. In: Barolli, L., Chen, H.-C., Enokido, T. (eds.) NBiS 2021. LNNS, vol. 313, pp. 50–58. Springer, Cham (2022). https://doi.org/10.1007/978-3-03084913-9 5 9. Kataoka, H., Duolikun, D., Sawada, A., Enokido, T., Takizawa, M.: Energy-aware server selection algorithms in a scalable cluster. In: Proceedings of the 30th International Conference on Advanced Information Networking and Applications, pp. 565–572 (2016) 10. Kataoka, H., Sawada, A., Dilawaer, D., Enokido, T., Takizawa, M.: Multi-level power consumption and computation models and energy-efficient server selection algorithms in a scalable cluster. In: Proceedings of the 19th International Conference on Network-Based Information Systems, pp. 210–217 (2016) 11. Kataoka, H., Nakamura, S., Duolikun, D., Enokido, T., Takizawa, M.: Multi-level power consumption model and energy-aware server selection algorithm. Int. J. Grid Util. Comput. 8(3), 201–210 (2017)

Autonomous Migration of Virtual Machines

29

12. Duolikun, D., Enokido, T., Takizawa, M.: Energy-efficient dynamic clusters of servers. In: Proceedings of the 8th International Conference on Broadband and Wireless Computing, Communication and Applications, pp. 253–260 (2013) 13. Duolikun, D., Enokido, T., Takizawa, M.: Static and dynamic group migration algorithms of virtual machines to reduce energy consumption of a server cluster. In: Nguyen, N.T., Kowalczyk, R., Xhafa, F. (eds.) Transactions on Computational Collective Intelligence XXXIII. LNCS, vol. 11610, pp. 144–166. Springer, Heidelberg (2019). https://doi.org/10.1007/978-3-662-59540-4 8 14. Duolikun, D., Enokido, T., Takizawa, M.: Simple algorithms for selecting an energy-efficient server in a cluster of servers. Int. J. Commun. Netw. Distrib. Syst. 21(1), 1–25 (2018) 15. Duolikun, D., Watanabe, R., Enokido, T., Takizawa, M.; An eco migration algorithm of virtual machines in a server cluster. In: Proceedings of IEEE the 32nd International Conference on Advanced Information Networking and Applications, pp. 189–196 (2018) 16. Duolikun, D., Enokido, T., Takizawa, M.: Energy-efficient group migration of virtual machines in a cluster. In: Barolli, L., Takizawa, M., Xhafa, F., Enokido, T. (eds.) AINA 2019. AISC, vol. 926, pp. 144–155. Springer, Cham (2020). https:// doi.org/10.1007/978-3-030-15032-7 12 17. Duolikun, D., Enokido, T., Barolli, L., Takizawa, M.: A monotonically increasing (MI) algorithm to estimate energy consumption and execution time of processes on a server. In: Proceedings of the 24th International Conference on Network-Based Information Systems, pp. 1–12 (2021) 18. Duolikun, D., Enokido, T., Barolli, L., Takizawa, M.: An energy-efficient algorithm to make virtual machines migrate in a server cluster. In: Barolli, L. (ed.) BWCCA 2021. LNNS, vol. 346, pp. 25–36. Springer, Cham (2022). https://doi.org/10.1007/ 978-3-030-90072-4 3 19. Duolikun, D., Enokido, T., Barolli, L., Takizawa, M.: An energy-efficient algorithm to make virtual machines migrate in a server cluster. Accepted at Proceedings of the 10th International Conference on Emerging Internet, Data and Web Technologies, pp. 130–141 (2022) 20. Duolikun, D., Enokido, T., Barolli, L., Takizawa, M.: An energy consumption model of servers to make virtual machines migrate. In: Proceedings of the 33rd International Conference on Advanced Information Networking and Applications, pp. 24–36 (2022) 21. Noaki, N., Saito, T., Duolikun, D., Enokido, T., Takizawa, M.: An energy-efficient algorithm for virtual machines to migrate considering migration time. In: Barolli, L., Takizawa, M., Enokido, T., Chen, H.-C., Matsuo, K. (eds.) BWCCA 2020. LNNS, vol. 159, pp. 341–354. Springer, Cham (2021). https://doi.org/10.1007/ 978-3-030-61108-8 34 22. Noguchi, K., Saito, T., Duolikun, D., Enokido, T., Takizawa, T.: An algorithm to select a server to minimize the total energy consumption of a cluster. In: Proceedings of the 15th International Conference on P2P, Parallel, Grid, Cloud and Internet Computing, pp. 18–28 (2020) 23. Natural Resources Defense Council (NRDS): Data center efficiency assessment - scaling up energy efficiency across the data center industry: Evaluating key drivers and barriers (2014). http://www.nrdc.org/energy/files/data-centerefficiency-assessment-IP.pdf 24. Watanabe, R., Duolikun, D., Takizawa, M.: Simple estimation and energy-aware migration models of virtual machines in a server cluster. Concurrency Comput. Pract. Exp. 30(21), e4771 (2018)

30

D. Duolikun et al.

25. Oma, R., Nakamura, S., Duolikun, D., Enokido, T., Takizawa, M.: An energyefficient model for fog computing in the Internet of Things (IoT). Internet Things 1–2, 14–26 (2018) 26. Oma, R., Nakamura, S., Enokido, T., Takizawa, M.: A tree-based model of energyefficient fog computing systems in IoT. In: Barolli, L., Javaid, N., Ikeda, M., Takizawa, M. (eds.) CISIS 2018. AISC, vol. 772, pp. 991–1001. Springer, Cham (2019). https://doi.org/10.1007/978-3-319-93659-8 92

An Indicate System for Danger Detection and Its Soldering Motion Analysis Tomoya Yasunaga1 , Tetsuya Oda2(B) , Kyohei Toyoshima1 , Yuki Nagai1 , Chihiro Yukawa1 , Kengo Katayama2 , and Leonard Barolli3 1

3

Graduate School of Engineering, Okayama University of Science (OUS), Okayama, 1-1 Ridaicho, Kita-ku, Okayama 700–0005, Japan {t22jm43sx,t22jm24jd,t22jm23rv,t22jm19st}@ous.jp 2 Department of Information and Computer Engineering, Okayama University of Science (OUS), 1-1 Ridaicho, Kita-ku, Okayama 700–0005, Japan {oda,katayama}@ice.ous.ac.jp Department of Information and Communication Engineering, Fukuoka Institute of Technology, 3-30-1 Wajiro-Higashi-ku, Fukuoka 811-0295, Japan [email protected]

Abstract. Soldering is prone to human error due to lack of concentration and corner-cutting caused by simple tasks. Therefore, it is expected that accidents can be reduced by having instructors give indication during dangerous actions or inappropriate postures during soldering iron work. In this paper, in order to solve these problems, we propose an indicate system for danger detection and carry out soldering motion analysis. Also, we show the experimental results for dangerous detection during soldering iron. The experimental results show that the proposed system has a good accuracy for detecting dangerous situations.

1

Introduction

Regardless of the industry, safety is given the highest priority in production. Safety is especially important because the cost of industrial accidents resulting from safety-related injuries, including human casualties, property damage, and production damage is extremely high. On the factory floor, the soldering is considered relatively easy to learn. In fact, in Japan, soldering training is provided in junior high school, high school, and university classes. Soldering is prone to human error due to lack of concentration and corner-cutting caused by simple tasks. Beginner soldering students are prone to human error due to inexperience, which is likely to lead to accidents such as poor contact of parts and burns. Therefore, it is expected that accidents can be reduced by having instructors give indication during dangerous [1–6] actions or inappropriate postures during soldering iron work. In this paper, in order to solve these problems, we propose an indicate system for danger detection. We carry out the analysis of soldering operation based on object detection [7–12], posture estimation [13–17] and smart speaker. We c The Author(s), under exclusive license to Springer Nature Switzerland AG 2022  L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 31–40, 2022. https://doi.org/10.1007/978-3-031-08819-3_4

32

T. Yasunaga et al.

also present experimental results of detecting dangerous situations when holding a soldering iron and dangerous postures during soldering based on image recognition. The structure of the paper is as follows. In Sect. 2, we describe the proposed system. In Sect. 3, we present the experimental results. Finally, conclusions and future work are given in Sect. 4.

2

Proposed System

The structure of the proposed system is shown in Fig. 1 and the system environment is shown in Fig. 2. • The following are body parts that can be recognized in real time by this system. – Soldering Iron – Soldering Iron’s Tip – Hand – Upper Body – Eyes – Legs Using these recognitions, the dangerous operation when using a soldering iron is determined. Furthermore, at the time of judgment, the smart speaker notifies the soldering iron user of the danger. In the method using a stereo camera, a finger is considered dangerous when it comes close to a hazardous area on the tip of the iron. The soldering iron and soldering iron tip are subject to object detection using YOLOv5 [18–20], and the soldering iron user’s hand is subject to hand tracking [21–24] using MediaPipe [25]. The BoundingBox of the soldering Iron’s tip derived from object recognition is considered the danger area, and the danger judgment determines whether the situation is dangerous or not by collision judgment considering the three-dimensional coordinates of the danger area and the fingertip. The threedimensional coordinates of the soldering irons’ tip consist of a, b, and z coordinates. The a and b are the diagonal coordinates of the starting point a (xa , ya ) and the end point b (xb , yb ) of a rectangle considering the soldering iron’s tip. The three-dimensional coordinates of the fingertip consist of two-dimensional coordinates (x, y) and z coordinates on the Z axis. The (x, y) are the fingertip coordinates based on hand tracking. The stereo camera of the proposed system can derive the distance between the camera and the object by triangulation from the parallax of the images captured by the two cameras. So, the z coordinates of the soldering iron’s tip and hand tracking indicate the distance from the camera to the fingertip obtained by triangulation. The method using UVC cameras estimates the pose of the entire body [26]. Since pose estimation using a single camera results in blind spots, we use three JetsonNanos and three UVC cameras for camera diversification. The coordinate data derived from each camera is aggregated on the Jetson Nano server and used

An Indicate System for Danger Detection and Its Soldering Motion Analysis

33

Fig. 1. Proposed system structure. Table 1. Examples of voice output at danger detection. Conditions

Content of output

Fingertip close to soldering iron dangerous area “It’s a dangerous way to hold it.” Soldering iron is away from hand

“Please hold the soldering iron.”

Upper body facing sideways

“It’s a dangerous posture.”

Crossed legs

“Please put your legs down.”

Eyes looking sideways

“Don’t look away.”

to determine the danger detection. In order to improve the recognition accuracy of the entire body, the height of each camera is kept constant, and the cameras are positioned at an angle of ±45 [deg.] with respect to the camera positioned in front. One of the methods using UVC cameras is to detect whether the upper body posture during soldering is dangerous or not. In the proposed system, the correct pose is one in which the body is parallel to the desk during soldering, and a dangerous pose is one in which the body is

34

T. Yasunaga et al.

Fig. 2. System environment

tilted forward or tilted more than a certain angle to the left or right. Poses are estimated based on the key points (x, y, z) obtained from skeletal estimation using MediaPipe. If the keypoints of the shoulders are closer than a certain distance to the front camera, the pose is judged as forward leaning. If the difference between the Z coordinates in the three-dimensional coordinates of both shoulders exceeds a certain value, the pose is determined as a dangerous pose with the upper body tilted to the left or right. Similarly for the lower body, using the y-coordinates below the ankles obtained from the pose estimation, the case where the y-coordinates are above a certain value is considered as a dangerous pose when soldering and danger detection is performed. In addition, for danger detection using both eyes, a camera located in the front is used to recognize the eyes. The center of the person is considered normal when the center coordinate is in the center. The person is considered in danger when the center coordinate is a certain distance away from the center. Google AIY VoiceKit V2 produces the voice output shown in Table 1 when judging danger detection. For example, when the user’s fingertip is close to the soldering iron’s tip hazardous area, Google AIY VoiceKit V2 tells the soldering iron user “You are holding the soldering iron in a dangerous way”.

An Indicate System for Danger Detection and Its Soldering Motion Analysis

35

(a) The state where there are no objects around (b) The correct way to hold the object and there and the correct way of holding. are tools around.

(c) Shows a dangerous holding position with no (d) Shows a dangerous holding position with objects around. tools around.

(e) Touching the dangerous area every 5 seconds (f) Touching the dangerous area every 5 seconds and there are no objects around. and there are tools around.

Fig. 3. Patterns of soldering iron holding.

3

Experimental Results

In Fig. 3 is shown the experimental environment and the way to hold the soldering. The experimental environment consists of a tabletop with no objects and

36

T. Yasunaga et al.

(a) Upper body twisting to the (b) Upper body twisting to the right. left.

(c) Bent forward

Fig. 4. Patterns of upper body pose.

(a) Eyes tracking.

(b) Vertical movement of both legs.

Fig. 5. Experimental environment for body dynamics.

a tabletop with work tools. The patterns of the experimental environment and the way of holding are described below. 1. Fig. 3a shows the state where there are no objects around and the correct way of holding. 2. Fig. 3b shows the correct way to hold the object and there are tools around. 3. Fig. 3c shows a dangerous holding position with no objects around. 4. Fig. 3d shows a dangerous holding position with tools around. 5. Fig. 3e shows a situation while touching the dangerous area every 5 s and there are no objects around. 6. Fig. 3f shows a situation while touching the dangerous area every 5 s and there are tools around. The dangerous area is defined as the area where the distance from the center of the soldering iron tip is 5.0 [cm]. In Fig. 3a and Fig. 3b are shown the percentage of correct answers, where each frame is the ratio of the judgment of danger detection with the total number of frames in 60 seconds. In Fig. 4 are shown the patterns of the upper body pose. Figure 5 shows the experimental

An Indicate System for Danger Detection and Its Soldering Motion Analysis

37

environment when the eyes and lower body are detected from the camera. We compare two states, one in which the subject remains still for 400 frames and the other in which the subject moves the body every 100 frames, to test whether the system is operating normally. As for the dynamic state, Fig. 5a is a case where the coordinates of the center of the eye are moved a certain distance to the left and right, and Fig. 5b is a case where the coordinates of the knee are moved a certain distance up and down. The experimental results of Fig. 3a to Fig. 3d and Fig. 4 are shown in Fig. 6. The correct rate in Fig. 6 is the averages of the correct rate measured 10 times for every 70 frames, where one frame is about 0.015 [sec.]. Figure 6a shows the correct holding of soldering iron with and without other work tools. The correct holding answer rate is 100.0 [%] and the danger holding rate of soldering iron with and without other work tools is about 83.2 [%] and 84.1 [%], respectively. From the experimental results, we can confirm that there is no difference in recognition accuracy due to differences in the environment. Figure 3e and Fig. 3f show the experimental scenario in which a finger is repeatedly moved into the danger zone of the soldering iron every 5 [sec.] for 60 [sec.]. Figure 7a and Fig. 7b show the results of the danger detection for Fig. 3e and Fig. 3f, respectively. Figure 7a shows that the results of danger detection are accurate with the movement of the finger. Figure 7b shows almost accurate results, but there are some errors because the soldering iron is recognized as Cutting Nippers in 35 [sec.] to 40 [sec.]. For danger detection based on pose estimation, the front camera is used to determine whether a pose during soldering is dangerous or not. A dangerous pose is defined as the upper body being bent forward or twisted sideways by more than 35 [◦ ]. The experiment is performed on the pattern of upper body poses shown in Fig. 4. Figure 6b shows the experimental results for each pattern of the upper body pose. It can be seen that all three patterns of the upper body have a high accuracy of about 95 [%] or more. Figure 8 shows the results of the experiment of Fig. 5 for dynamic and static stater of the body. Figure 8a detects eye movements and the dynamic state can be confirmed every 100 frames, but even in the static state for 100 frames, it can be seen to be moving. Since blinking may be detected incorrectly, it is necessary to improve the system to take blinking into account. Figure 8b shows that the system detected leg movements, and since no false positives were observed in both the moving and stationary states, it is considered to be useful for the proposed system.

38

T. Yasunaga et al.

Correct Holding

Danger Holding

100

100

80

Correct Rate

Correct Rate

80

60

60

40

40

20

20

0

0 (a)

(b)

(c)

(d)

(a)

Type of Holding and Environment

(b)

(c)

Detection Rate of Danger Pose

(a) The soldering iron holding.

(b) The upper body pose.

Danger Detection [unit]

Danger Detection [unit]

Fig. 6. Experimental results for soldering iron holding and upper body pose.

1

0 0

5 10 15 20 25 30 35 40 45 50 55 60

Times [fps]

1

0 0

5 10 15 20 25 30 35 40 45 50 55 60

Times [fps]

Fig. 7. Experimental results during touching dangerous area.

Fig. 8. Experimental results for dynamic states and static states of the body.

An Indicate System for Danger Detection and Its Soldering Motion Analysis

4

39

Conclusions

In this paper, we proposed an indicate system for danger detection and carried out soldering motion analysis. From the experimental results, we found that object detection and posture estimation can determine and indicate dangerous situations during the soldering iron usage. In the future, we would like to improve the proposed system by adding new functions. Acknowledgement. This work was supported by JSPS KAKENHI Grant Number JP20K19793.

References 1. Yasunaga, T., et al.: Object detection and pose estimation approaches for soldering danger detection. In: Proceedings of the IEEE 10-th Global Conference on Consumer Electronics, pp. 776–777 (2021) 2. Yasunaga, T., et al.: A soldering motion analysis system for danger detection considering object detection and attitude estimation. In: Proceedings of the 10-th International Conference on Emerging Internet, Data & Web Technologies, pp. 301–307 (2022) 3. Toyoshima, K., et al.: Proposal of a haptics and LSTM based soldering motion analysis system. In: Proceedings of the IEEE 10-th Global Conference on Consumer Electronics, pp. 1–2 (2021) 4. Hirota, Y., et al.: Proposal and experimental results of an ambient intelligence for training on soldering iron holding. In: Proceedings of BWCCA 2020, pp. 444–453 (2020) 5. Oda, T., et al.: Design and Implementation of an IoT-based E-learning Testbed. Int. J. Web Grid Serv. 13(2), 228–241 (2017) 6. Liu, Y., et al.: Design and implementation of testbed using IoT and P2P technologies: improving reliability by a fuzzy-based approach. Int. J. Commun. Netw. Distrib. Syst. 19(3), 312–337 (2017) 7. Papageorgiou, C., et al.: A general framework for object detection. In: The IEEE 6th International Conference on Computer Vision, pp. 555–562 (1998) 8. Felzenszwalb, P., et al.: Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. 32(9), 1627–1645 (2009) 9. Obukata, R., et al.: Design and evaluation of an ambient intelligence testbed for improving quality of life. Int. J. Space-Based Situated Comput. 7(1), 8–15 (2017) 10. Oda, T., et al.: Design of a deep Q-network based simulation system for actuation decision in ambient intelligence. In: Proceedings of AINA 2019, pp. 362–370 (2019) 11. Obukata, R., et al.: Performance evaluation of an am I testbed for improving QoL: evaluation using clustering approach considering distributed concurrent processing. In: Proceedings of IEEE AINA 2017, pp. 271–275 (2017) 12. Yamada, M., et al.: Evaluation of an IoT-based e-learning testbed: performance of OLSR protocol in a NLoS environment and mean-shift clustering approach considering electroencephalogram data. Int. J. Web Inf. Syst. 13(1), 2–13 (2017) 13. Toshev, A., Szegedy, C.: DeepPose: human pose estimation via deep neural networks. In: Proceedings of the 27-th IEEE/CVF Conference on Computer Vision and Pattern Recognition (IEEE/CVF CVPR 2014), pp. 1653–1660 (2014)

40

T. Yasunaga et al.

14. Haralick, R., et al.: Pose estimation from corresponding point data. IEEE Trans. Syst. 19(6), 1426–1446 (1989) 15. Fang, H., et al.: RMPE: regional multi-person pose estimation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2334–2343 (2017) 16. Xiao, B., Wu, H., Wei, Y.: Simple baselines for human pose estimation and tracking. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11210, pp. 472–487. Springer, Cham (2018). https://doi.org/10.1007/978-3030-01231-1 29 17. Martinez, J., et al.: A simple yet effective baseline for 3D human pose estimation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2640–2649 (2017) 18. Redmon, J., et al.: You only look once: unified, real-time object detection. In: Proceedings of the 29-th IEEE/CVF Conference on Computer Vision and Pattern Recognition (IEEE/CVF CVPR 2016), pp. 779–788 (2016) 19. Zhou, F., et al.: Safety helmet detection based on YOLOv5. In: The IEEE International Conference on Power Electronics, Computer Applications (ICPECA), pp. 6–11 (2021) 20. Yu-Chuan, B., et al.: Using improved YOLOv5s for defect detection of thermistor wire solder joints based on infrared thermography. In: The 5th International Conference on Automation, Control and Robots (ICACR), pp. 29–32 (2021) 21. Zhang, F., et al.: MediaPipe hands: on-device real-time hand tracking. arXiv preprint arXiv:2006.10214 (2020) 22. Shin, J., et al.: American sign language alphabet recognition by extracting feature from hand pose estimation. Sensors 21(17), 5856 (2021) 23. Hirota, Y., et al.: Proposal and experimental results of a DNN based real-time recognition method for Ohsone style fingerspelling in static characters environment. In: Proceedings of the IEEE 9-th Global Conference on Consumer Electronics, pp. 476–477 (2020) 24. Erol, A., et al.: Vision-based hand pose estimation: a review. Comput. Vis. Image Underst. 108, 52–73 (2007) 25. Lugaresi, C., et al.: MediaPipe: a framework for building perception pipelines. arXiv preprint arXiv:1906.08172 (2019) 26. Micilotta, A.S., Ong, E.-J., Bowden, R.: Real-time upper body detection and 3D pose estimation in monoscopic images. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3953, pp. 139–150. Springer, Heidelberg (2006). https://doi.org/10.1007/11744078 11

A Movement Adjustment Method for LiDAR Based Mobile Area Decision: Improving Control for AAV Mobility Nobuki Saito1 , Tetsuya Oda2(B) , Chihiro Yukawa1 , Kyohei Toyoshima1 , Aoto Hirata1 , and Leonard Barolli3 1

3

Graduate School of Engineering, Okayama University of Science (OUS), 1-1 Ridaicho, Kita-ku, Okayama 700–0005, Japan {t21jm01md,t22jm19st,t22jm24jd,t21jm02zr}@ous.jp 2 Department of Information and Computer Engineering, Okayama University of Science (OUS), 1-1 Ridaicho, Kita-ku, Okayama 700–0005, Japan [email protected] Department of Information and Communication Engineering, Fukuoka Institute of Technology, 3-30-1 Wajiro-higashi, Higashi-ku, Fukuoka 811-0295, Japan [email protected]

Abstract. The Deep Q-Network (DQN) is one of the deep reinforcement learning algorithms, which uses deep neural network structure to estimate the Q-value in Q-learning. In the previous work, we designed and implemented a DQN-based Autonomous Aerial Vehicle (AAV) testbed and proposed a Tabu List Strategy based DQN (TLS-DQN). In this paper, we propose a B-spline curve based movement adjustment method to improve the DQN-based AAV mobility control method. The performance evaluation results show that the proposed method can consider the obstacles and decrease the movement fluctuations.

1

Introduction

The Unmanned Aerial Vehicle (UAV) is expected to be used in different fields such as aerial photography, transportation, search and rescue of humans, inspection, land surveying, observation and agriculture. Autonomous Aerial Vehicle (AAV) [1] has the ability to operate autonomously without human control and is expected to be used in a variety of fields, similar to UAV. So far many AAVs [2–4] are proposed and used practically. However, existing autonomous flight systems are designed for outdoor use and rely on location information by the Global Navigation Satellite System (GNSS) or others. On the other hand, in an environment where it is difficult to obtain position information from GNSS, it is necessary to determine a path without using position information. Therefore, autonomous movement control is essential to achieve operations that are independent of the external environment, including non-GNSS environments such as indoor, tunnel and underground. c The Author(s), under exclusive license to Springer Nature Switzerland AG 2022  L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 41–53, 2022. https://doi.org/10.1007/978-3-031-08819-3_5

42

N. Saito et al.

In [5–7] the authors consider Wireless Sensor and Actuator Networks (WSANs), which can act autonomously for disaster monitoring. A WSAN consists of wireless network nodes, all of which have the ability to sense events (sensors) and perform actuation (actuators) based on the sensing data collected by the sensors. WSAN nodes in these applications are nodes with integrated sensors and actuators that have the high processing power, high communication capability, high battery capacity and may include other functions such as mobility. The application areas of WSAN include AAV [8], Autonomous Surface Vehicle (ASV) [8], Heating, Ventilation, Air Conditioning (HVAC) [9], Internet of Things (IoT) [10], Ambient Intelligence (AmI) [11], and so on. Deep reinforcement learning [13] is an intelligent algorithm that is effective in controlling autonomous robots such as AAV. Deep reinforcement learning is an approximation method using deep neural network for value function and policy function in reinforcement learning. Deep Q-Network (DQN) is a method of deep reinforcement learning using Convolution Neural Network (CNN) as a function approximation of Q-values in the Q-learning algorithm [13]. DQN combines the neural fitting Q-iteration [14] and experience replay [15], shares the hidden layer of the action value function for each action pattern and can stabilize learning even with nonlinear functions such as CNN [16]. However, there are some points where learning is difficult to progress for problems with complex operations and rewards, or problems where it takes a long time to obtain a reward. In this paper, we propose a Basis-spline (B-spline) curve based movement adjustment method to improve the DQN-based AAV mobility control method [17,18]. Also, we present the simulation results for AAV control using TLS-DQN [19–21] considering an indoor single-path environment with a staircase. The structure of the paper is as follows. In Sect. 2, we show the DQN based AAV testbed. In Sect. 3, we describe the proposed method. In Sect. 4, we discuss the performance evaluation. Finally, conclusions and future work are given in Sect. 5.

2

DQN Based AAV Testbed

In this section, we discuss quadrotor for AAV and DQN for AAV mobility. 2.1

Quadrotor for AAV

For the design of AAV, we consider a quadrotor, which is a type of multicopter. Multicopter is high maneuverable and can operate in places that are difficult for people to enter, such as disaster areas and dangerous places. It also has the advantage of not requiring space for takeoffs and landings and being able to stop at mid-air during the flight, therefore enabling activities at fixed points. In Fig. 1 is shown a snapshot of the quadrotor used for designing and implementing the AAV testbed. The quadrotor frame is mainly composed of polyvinyl

A Movement Adjustment Method for LiDAR Based Mobile Area Decision

43

Fig. 1. Snapshot of AAV. Table 1. Components of quadrotor. Components

Model

Propeller

15 × 5.8

Motor

MN3508 700 kv

Electric speed controller

F45A 32bitV2

Flight controller

Pixhawk 2.4.8

Power distribution Board MES-PDB-KIT Li-Po battery

22.2 v 12000 mAh XT90

Mobile battery

Pilot Pro 2 23000 mAh

ToF ranging sensor

VL53L0X

Raspberry Pi

3 model B plus

PVC Pipe

VP20

Acrylic plate

5 mm

Fig. 2. AAV control system.

chloride (PVC) pipe and acrylic plate. The components for connecting the battery, motor, sensor and so on to the frame are created using an optical 3D printer. Table 1 shows the components in the quadrotor. The size specifications of the quadrotor (including the propeller) are length 87 [cm], width 87 [cm], height 30 [cm] and weight 4259 [g]. In Fig. 2 is shown the AAV control system. The Raspberry Pi reads saved data of the best episode when carrying out the simulations by DQN and uses telemetry communication to send commands such as up, down, forward, back,

44

N. Saito et al.

left, right and stop to the flight controller. Also, multiple Time-of-Flight (ToF) range sensors using Inter-Integrated Circuit (I2 C) communication and General-Purpose Input Output (GPIO) are used to acquire and save flight data. The Flight Controller (FC) is a component that calculates the optimum motor rotation speed for flight based on the information sent from the built-in acceleration sensor and gyro sensor. The Electronic Speed Controller (ESC) is a part that controls the rotation speed of the motor in response to commands from FC. Through these sequences, AAV behaves and reproduces movement in simulation. 2.2

DQN for AAV Mobility

The DQN for moving control of AAV structure is shown in Fig. 3. In this work, we use the Deep Belief Network (DBN), because the computational complexity is smaller than CNN for DNN part in DQN. The environment is set as vi . At each step, the agent selects an action at from the action sets of the mobile actuator nodes and observes a position vt from the current state. The change of the mobile actuator node score rt was regarded as the reward for the action. For the reinforcement learning, we can complete all of these mobile actuator nodes sequences mt as Markov decision process directly, where sequences of observations and actions are mt = v1 , a1 , v2 , . . . , at−1 , vt . A method known as experience replay is used to store the experiences of the agent at each timestep, et = (mt , at , rt , mt+1 ) in a dataset D = e1 , . . . , eN , cached over many episodes into a Experience Memory. By defining the discounted reward for the by a Tfuture  factor γ, the sum of the future reward until the end would be Rt = t =t γ t −t rt . T means the termination time-step of the mobile actuator nodes. After running experience replay, the agent selects and executes an action according to an greedy strategy. Since using histories of arbitrary length as inputs to a neural network can be difficult, Q-function instead works on fixed length format of histories produced by a function φ. The target was to maximize the action value function Q∗ (m, a) = maxπ E[Rt |mt = m, at = a, π], where π is the strategy for selecting of best action. From the Bellman equation (see Eq. (1)), it is possibel to maximize the expected value of r + γQ∗ (m , a ), if the optimal value Q∗ (m , a ) of the sequence at the next time step is known. Q∗ (m , a ) = Em ∼ξ [r + γa maxQ∗ (m , a )|m, a].

(1)

By not using iterative updating method to optimize the equation, it is common to estimate the equation by using a function approximator. Q-network in DQN is a neural network function approximator with weights θ and Q(s, a; θ) ≈ Q∗ (m, a). The loss function to train the Q-network is shown in Eq. (2): Li (θi ) = Es,a∼ρ(· ) [(yi − Q(s, a; θi ))2 ].

(2)

The yi is the target, which is calculated by the previous iteration result θi−1 . The ρ(m, a) is the probability distribution of sequences m and a. The gradient of the loss function is shown in Eq. (3): ∇θi Li (θi ) = Em,a∼ρ(· );s ∼ξ [(yi − Q(m, a; θi ))∇θi Q(m, a; θi )].

(3)

A Movement Adjustment Method for LiDAR Based Mobile Area Decision

45

Fig. 3. DQN for AAV mobility control.

We consider tasks in which an agent interacts with an environment. In this case, the AAV moves step by step in a sequence of observations, actions and rewards. We took in consideration AAV mobility and consider 7 mobile patterns (up, down, forward, back, left, right, stop). In order to decide the reward function, we considered distance between AAV and Obstacle (DAO) parameter. The initial weights values are assigned as Normal Initialization. The input layer is using AAV and the position of destination, total reward values in Experience Memory and AAV movements patterns. The hidden layer is connected with 256 rectifier units in Rectified Linear Units (ReLU) [22]. The output Q-values are the AAV movement patterns.

3

Proposed Method

In this section, we discuss the AAV mobility control method for DQN-based AAV. 3.1

LiDAR Based Mobile Area Decision Method

The proposed method decides the destination in TLS-DQN within the considered area based on the point cloud obtained by LiDAR and reduces the setting operation for the destination set manually by humans in TLS-DQN. In Algorithm 1, we consider as inputs the coordinates list of obstacles (distance, angle) obtained by LiDAR and the coordinates of LiDAR placement. The output is the Destination (X, Y ), which may be local destination or global destination. The global destination indicates the destination in the considered area, and the local destination indicates the target passage points until the global destination. The Z-coordinate of destination is the median of the movable range in the Z-axis for the destination. In the proposed method, the destination is continuously decided by letting the LiDAR placement be the coordinate of reached destination when the AAV reached the destination.

46

N. Saito et al.

Algorithm 1. LiDAR Based Mobile Area Decision Method. Input: P oint Cloud List ← The coordinates list of obstacles (distance, angle) obtained by LiDAR (xLiDAR , yLiDAR ) ← The coordinates of LiDAR Placement. Output: Destination (X, Y ). 1: for i = 0 to 360 do 2: xP oint Cloud List [i] ← P oint Cloud List[i][0] × cos(P oint Cloud List[i][1]). 3: yP oint Cloud List [i] ← P oint Cloud List[i][0] × sin(P oint Cloud List[i][1]). 4: if P oint Cloud List[i][0] > Any Distance then 5: Distant P oint Cloud[i] ← (xP oint Cloud List [i], yP oint Cloud List [i]). 6: (xmin , xmax ) ← Min. and Max. value for X-axis in the Distant P oint Cloud. 7: (ymin , ymax ) ← Min. and Max. value for Y-axis in the Distant P oint Cloud. 8: (xcenter , ycenter ) ← ( xmin +2 xmax , ymin +2 ymax ). 9: f lag ← 0. 10: for x = xLiDAR to xcenter do − yLiDAR ) × (x − xLiDAR ) + yLiDAR . 11: y ← ( xycenter center − xLiDAR 12: for i  = 0 to 360 do (x − xP oint Cloud List [i])2 + (y − yP oint Cloud List [i])2 > Any Distance 13: if then 14: Destination ← (x, y). 15: else 16: f lag ← 1. 17: break 18: if f lag = 0 then 19: Destination is local destination. 20: else 21: Destination is global destination.

Algorithm 2. Tabu List for TLS-DQN. Require: The coordinate with the highest evaluated value in the section is (x, y, z). 1: if (xbef ore ≤ xcurrent ) ∧ (xcurrent ≤ x) then 2: tabu list ⇐ ((xmin ≤ xbef ore ) ∧ (ymin ≤ ymax ) ∧ (zmin ≤ zmax )) 3: else if (xbef ore ≥ xcurrent ) ∧ (xcurrent ≥ x) then 4: tabu list ⇐ ((xbef ore ≤ xmax ) ∧ (ymin ≤ ymax ) ∧ (zmin ≤ zmax )) 5: else if (ybef ore ≤ ycurrent ) ∧ (ycurrent ≤ y) then 6: tabu list ⇐ ((xmin ≤ xmax ) ∧ (ymin ≤ ybef ore ) ∧ (zmin ≤ zmax )) 7: else if (ybef ore ≥ ycurrent ) ∧ (ycurrent ≥ y) then 8: tabu list ⇐ ((xmin ≤ xmax ) ∧ (ybef ore ≤ ymax ) ∧ (zmin ≤ zmax )) 9: else if (zbef ore ≤ zcurrent ) ∧ (zcurrent ≤ z) then 10: tabu list ⇐ ((xmin ≤ xmax ) ∧ (ymin ≤ ymax ) ∧ (zmin ≤ zbef ore )) 11: else if (zbef ore ≥ zcurrent ) ∧ (zcurrent ≥ z) then 12: tabu list ⇐ ((xmin ≤ xmax ) ∧ (ymin ≤ ymax ) ∧ (zbef ore ≤ zmax ))

3.2

TLS-DQN

The idea of the Tabu List Strategy (TLS) is motivated from Tabu Search (TS) [23] to achieve an efficient search for various optimization problems by

A Movement Adjustment Method for LiDAR Based Mobile Area Decision

47

prohibiting movements to previously visited search area in order to prevent getting stuck in local optima. ⎧ 3 ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨

(if (xcurrent = xglobal destinations ) ∧ (ycurrent = yglobal destinations ) ∧ (zcurrent = zglobal destinations )) ∨ (((xbef ore < xcurrent ) ∧ (xcurrent ≤ xlocal destinations )) ∨ ((xbef ore > xcurrent ) ∧ (xcurrent ≥ xlocal destinations )) r= ∨ ((ybef ore < ycurrent ) ∧ (ycurrent ≤ ylocal destinations )) ⎪ ⎪ ⎪ ⎪ ∨ ((ybef ore > ycurrent ) ∧ (ycurrent ≥ ylocal destinations )) ⎪ ⎪ ⎪ ⎪ ∨ ((zbef ore < zcurrent ) ∧ (zcurrent ≤ zlocal destinations )) ⎪ ⎪ ⎪ ⎪ ∨ ((zbef ore > zcurrent ) ∧ (zcurrent ≥ zlocal destinations ))). ⎪ ⎪ ⎩ −1 (else).

(4)

In this paper, the reward value for DQN is decided by Eq. (4), where “x”, “y” and “z” means X-axis, Y -axis and Z-axis, respectively. The current means the current coordinates of the AAV in the DQN, and the before means the coordinates before moving the action. The considered area is partitioned based on the local destination or global destination and a destination is set in each area. The tabu list in TLS is used when a DQN selects an action randomly or determined a reward for the action. If the tabu list includes the area in the direction of movement, the DQN will reselect the action. Also, if the reward is “3”, the prohibited area is added to the tabu list based on the rule shown in Algorithm 2. The search by TLS-DQN is done in a wider range and is better than the search by random movement direction. 3.3

Movement Adjustment Method

The movement adjustment method is used for reducing movement fluctuations caused by TLS-DQN. The Algorithm 3 inputs the movement of coordinates (X, Y , Z) in the episode of Best derived by TLS-DQN and generates the Adjustment Point Coordinates List. In Algorithm 3, the Number of divided list indicates the number of divisions to the coordinate movements; the N umber of coordinates indicates how many coordinates are included in the Divided List; and the (xcneter , ycneter , zcneter ) indicates the center coordinates derived from the maximum and minimum values of coordinates in X-axis, Y -axis and Z-axis included in the Divided List. In addition, the B-spline curve based movement adjustment method uses the Adjustment point coordinates list as the control point for a deriving movement that is continuous and smooth. It also determines an executable path or a non-executable path by detecting the collision between the B-spline curve and obstacles.

48

N. Saito et al.

Algorithm 3. Movement Adjustment Decision. Input: Movement Coordinates ← The movement of coordinates (X, Y , Z) by TLSDQN Output: Adjustment Point Coordinates List. 1: Number of divided list ← Any number. of Iterations in TLS-DQN . 2: Number of coordinates ← NumberNumber of divided list 3: i ← 0, j ← 0 4: for k = 0 to Number of coordinates in Movement Coordinates. do 5: Divided List[j] ← M ovement Coordinates[k]. 6: j ← j + 1. 7: if j ≥ Number of coordinates then 8: (xmin , xmax , ymin , ymax , zmin , zmax ) ← Min. and Max. values for each axis in the Divided List. 9: (xcenter , ycenter , zcenter ) ← ( xmin +2 xmax , ymin +2 ymax , zmin +2 zmax ) 10: Adjustment Point Coordinates List[i] ← (xcenter , ycenter , zcenter ). 11: i ← i + 1, j ← 0

(a) From the initial place- (b) From the corner space to (c) From the global destinament to the corner space. the global destination. tion to the corner space.

Fig. 4. Snapshot of the considered area.

4

Performance Evaluation

In this section, we discuss the experimental results of LiDAR based decision method, the simulation results of TLS-DQN and the movement adjustment method. 4.1

Results of LiDAR Based Decision Method

The target environment is an indoor single-path environment with a staircase. Figure 4 shows snapshots of the area used in the simulation scenario and was taken on the ground floor of Building C5 at Okayama University of Science, Japan. In Fig. 4, Fig. 5 and Fig. 6, the blue filled area indicates the floor surface, the red filled area indicates the corner space and the green filled area indicates the staircase space. Figure 5 visualize the experimental results of the LiDAR based mobile area decision method. Figure 5 shows the visualization results of LiDAR based decision method, the obstacle obtained by LiDAR and the decided destination. Figure 6 shows the considered area based on the actual measurements of Fig. 5. The experimental results show that the path and the global destination is decided at the end of the path through the staircase space.

A Movement Adjustment Method for LiDAR Based Mobile Area Decision

49

Fig. 5. Visualization results of LiDAR based decision method.

Fig. 6. Considered area for simulation. Table 2. Simulation parameters of TLS-DQN. Parameters

Values

Number of episode

50000

Number of iteration

2000

Number of hidden layers

3

Number of hidden units

15

Initial weight value

Normal initialization

Activation function

ReLU

Action selection probability () 0.999 − (t/Number of episode) (t = 0, 1, 2, . . ., Number of episode) Learning rate (α)

4.2

0.04

Discount rate (γ)

0.9

Experience memory size

300 × 100

Batch size

32

Number of AAV

1

Simulation Results of TLS-DQN

We consider for simulations the operations such as takeoffs, flights and landings between the initial position and the destination decided by the LiDAR based decision method. The target environment is an indoor single-path environment including a staircase. Table 2 shows the parameters used in the simulations.

50

N. Saito et al.

Fig. 7. Simulation results of rewards.

Fig. 8. Visualization results of TLS-DQN and movement adjustment method. Table 3. Movement distance. Method

XYZ

XY plane YZ plane

Best episode of TLS-DQN

1195.000 792.000

787.000

B-spline curve adjustment to best episode

470.932 370.417

377.653

Number of divided list = 45

292.110 261.175

262.532

B-spline curve adjustment with control points = 45

290.851 260.137

262.670

Number of divided list = 15

270.746 249.551

249.607

B-spline curve adjustment with control points = 15

272.366 252.179

248.249

Figure 7 shows the change in reward value of the action in each iteration for Worst, Median, and Best episodes in TLS-DQN. For Best episodes, the reward value is increased much more than Median episodes. While, for Worst episodes, the reward value is decreased. 4.3

Results of Movement Adjustment Method

Figure 8 shows the visualization results on the XY and Y Z planes for the movement of the Best episodes in TLS-DQN and the results of the movement adjustment method when the Number of divided list is 45, 15 and 14, respectively. Also, Fig. 8 shows visualization results of the B-spline curve based movement adjustment with control point when the Number of divided list is 45, 15, and 14, respectively. From the visualization results, the TLS-DQN has reached the destination. Also, the movement by the B-spline curve based movement adjustment with 14 control points considered a non-executable movement because it detects the collision with an obstacle. On the other hand, the movement by B-spline curve based movement adjustment with 15 or 45 control points is an

A Movement Adjustment Method for LiDAR Based Mobile Area Decision

51

executable movement because it does not pass through an obstacle. Table 3 shows the movement distance derived from the total Euclidean distances between each coordinate. In the case of the B-spline curve-based movement adjustment with 15 control points, the movement distance in XYZ is reduced by 77.207 [%] compared to the Best Episode of TLS-DQN, by 42.164 [%] compared to the B-spline curve based movement adjustment to Best Episode of TLS-DQN, by 6.355 [%] compared to the B-spline curve-based movement adjustment with 45 control points. Also, compared to the case when the Number of divided list is 15, the movement distance has increased in XY Z and XY plane. However, when the Number of divided list is 15, it is a non-executable path, whereas the B-spline based movement adjustment with 15 control points is an executable path. The performance evaluation shows that the movement adjustment can decrease the movement distance and fluctuations. Also, the B-spline curve based movement adjustment considers obstacles and can derive continuous and smooth movement.

5

Conclusions

In this paper, we proposed a B-spline curve based movement adjustment method to improve the DQN-based AAV mobility control method. Also, we presented the simulation results for AAV control using TLS-DQN considering an indoor single-path environment with a staircase. From performance evaluation results, we conclude as follows. • The proposed method can decide the mobile area and destination based on LiDAR and TLS-DQN. • The visualization results of the movement show that the TLS-DQN can reach the destination and the proposed method can derive smooth movement. • The proposed method is a good approach for indoor single-path environments. In the future, we would like to improve the TLS-DQN for AAV mobility by considering different scenarios. Acknowledgement. This work was supported by JSPS KAKENHI Grant Number JP20K19793 and Grant for Promotion of OUS Research Project (OUS-RP-20-3).

References 1. St¨ ocker, C., et al.: Review of the current state of UAV regulations. Remote Sens. 9(5), 1–26 (2017) 2. Popovi´c, M., et al.: An informative path planning framework for UAV-based terrain monitoring. Auton. Robots 44, 889–911 (2020) 3. Nguyen, H., et al.: LAVAPilot: lightweight UAV trajectory planner with situational awareness for embedded autonomy to track and locate radio-tags. arXiv:2007.15860, pp. 1–8 (2020) 4. Saito, N., et al.: Design and implementation of a DQN based AAV. In: Proceedings of The 15th International Conference on Broad-Band Wireless Computing, Communication and Applications (BWCCA-2020), pp. 321–329 (2020)

52

N. Saito et al.

5. Oda, T., et al.: Design and implementation of a simulation system based on deep Q-network for mobile actor node control in wireless sensor and actor networks. In: Proceedings of The 31th IEEE International Conference on Advanced Information Networking and Applications Workshops (IEEE AINA-2017), pp. 195–200 (2017) 6. Oda, T., et al.: Performance evaluation of a deep Q-network based simulation system for actor node mobility control in wireless sensor and actor networks considering three-dimensional environment. In: Proceedings of The 9th International Conference on Intelligent Networking and Collaborative Systems (INCoS-2017), pp. 41–52 (2017) 7. Oda, T., et al.: A deep Q-network based simulation system for actor node mobility control in WSANs considering three-dimensional environment: a comparison study for normal and uniform distributions. In: Proceedings of CISIS-2018, pp. 842–852 (2018) 8. Saito, N., et al.: Proposal and evaluation of a Tabu list based DQN for AAV mobility. In: Proceedings of The 9th International Conference on Emerging Internet, Data and Web Technologies (EIDWT-2021), pp. 156–167 (2021) 9. Moulton, J., et al.: An autonomous surface vehicle for long term operations. In: Proceedings of MTS/IEEE OCEANS, pp. 1–10 (2018) 10. Oda, T., et al.: Design of a deep Q-network based simulation system for actuation decision in ambient intelligence. In: Proceedings of the 33rd International Conference on Advanced Information Networking and Applications (AINA-2019), pp. 362–370 (2019) 11. Oda, T., et al.: Design and implementation of an IoT-based E-learning testbed. Int. J. Web Grid Serv. 13(2), 228–241 (2017) 12. Hirota, Y., et al.: Proposal and experimental results of an ambient intelligence for training on soldering iron holding. In: Proceedings of the 15th International Conference on Broadband and Wireless Computing, Communication and Applications (BWCCA-2020), pp. 444–453 (2020) 13. Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015) 14. Riedmiller, M.: Neural fitted Q iteration – first experiences with a data efficient neural reinforcement learning method. In: Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds.) ECML 2005. LNCS (LNAI), vol. 3720, pp. 317–328. Springer, Heidelberg (2005). https://doi.org/10.1007/11564096 32 15. Lin, L.J.: Reinforcement learning for robots using neural networks. In: Proceedings of Technical Report, DTIC Document (1993) 16. Lange, S., Riedmiller, M.: Deep auto-encoder neural networks in reinforcement learning. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN-2010), pp. 1–8 (2010) 17. Saito, N., et al.: Performance evaluation of a DQN-based autonomous aerial vehicle mobility control method in an indoor single-path environment with a staircase. In: Proceedings of the 10th International Conference on Emerging Internet, Data and Web Technologies (EIDWT-2022), pp. 417–429 (2022) 18. Saito, N., et al.: A movement adjustment method for DQN-based autonomous aerial vehicle. In: Proceedings of the 13th International Conference on Intelligent Networking and Collaborative Systems (INCoS-2021), pp. 136–148 (2021) 19. Saito, N., et al.: A Tabu list strategy based DQN for AAV mobility in indoor singlepath environment: implementation and performance evaluation. Internet Things 14, 100394 (2021)

A Movement Adjustment Method for LiDAR Based Mobile Area Decision

53

20. Saito, N., et al.: A LiDAR based mobile area decision method for TLS-DQN: improving control for AAV mobility. In: Proceedings of the 16th International Conference on P2P, Parallel, Grid, Cloud and Internet Computing (3PGCIC-2021), pp. 30–42 (2021) 21. Saito, N., et al.: Simulation results of a DQN based AAV testbed in corner environment: a comparison study for normal DQN and TLS-DQN. In: Proceedings of the 15th International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing (IMIS-2021), pp. 156–167 (2021) 22. Glorot, X., et al.: Deep sparse rectifier neural networks. In: Proceedings of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS-2011), pp. 315–323 (2011) 23. Glover, F.: Tabu search - part I. ORSA J. Comput. 1(3), 190–206 (1989)

Design and Implementation of a Haptics Based Soldering Education System Kyohei Toyoshima1 , Tetsuya Oda2(B) , Tomoya Yasunaga1 , Chihiro Yukawa1 , Yuki Nagai1 , Nobuki Saito1 , and Leonard Barolli3 1

Graduate School of Engineering, Okayama University of Science (OUS), Okayama, 1-1 Ridaicho, Kita-ku, Okayama 700–0005, Japan {t22jm24jd,t22jm43sx,t22jm19st,t22jm23rv,t21jm01md}@ous.jp 2 Department of Information and Computer Engineering, Okayama University of Science (OUS), 1-1 Ridaicho, Kita-ku, Okayama 700–0005, Japan [email protected] 3 Department of Information and Communication Engineering, Fukuoka Insitute of Technology, 3-30-1 Wajiro-Higashi-ku, Fukuoka 811-0295, Japan [email protected]

Abstract. The soldering techniques are one of the industrial techniques required in electronic device manufacturing plants. However, a framework for quantifying soldering techniques has not been established, therefore it hard to evaluate the training of trainees. Also, The safety is very important in the education of beginners. The haptics is able to transmit the power generated in the virtual space to the manipulator. Therefore, we perform soldering virtual training based on haptics and analyze the soldering motion based on Long Short-Term Memory (LSTM) using training data in a virtual space. In this paper, we propose and evaluate a soldering education system based on haptics. The experimental results show that the proposed system is able to detect dangerous movements in the soldering motion.

1

Introduction

The soldering techniques are one of the industrial techniques required in electronic device manufacturing plants. They are very important techniques because they affect the product quality. For soldering training, it is hard for trainers to instruct a large number of trainees at the same time because it may cause accidents in production sites. Also, a framework for quantifying soldering techniques has not been established, therefore it hard to evaluate the training of trainees. The safety is very important in the education of beginners. Furthermore, technical training for beginners requires the explanation of the detailed work procedures and the burden on the trainer is increased by monitoring process to prevent accidents. On the other hand, the application of haptics to virtual space can reduce the occurrence of accidents during training. The haptics is able to transmit the power generated in the virtual space to the manipulator. The haptics enables training similar to that in real space and is using a wide range c The Author(s), under exclusive license to Springer Nature Switzerland AG 2022  L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 54–64, 2022. https://doi.org/10.1007/978-3-031-08819-3_6

Design and Implementation of a Haptics Based Soldering Education System

55

of fields including medicine [1,2], welfare [3], sports [4] and so on. [5–9] Therefore, we implement a haptics based virtual space that can perform soldering and collect the soldering techniques training data. We analyze the soldering motion based on Long Short-Term Memory (LSTM) [10–15]. The LSTM is a method of Recurrent Neural Network (RNN) [16–18], which is used for natural language processing [19–22] and acoustic analysis [23] since it can predict time series. In addition, it can learn long-term dependencies. In this paper, we propose and evaluate a haptics based soldering education system. For evaluation of the proposed system, we perform the detection of anomalies for motions of the soldering technique. The structure of the paper is as follows. In Sect. 2, we present the proposed system. In Sect. 3, we explain the experimental scenario. In Sect. 4, we describe the experimental results. Finally, conclusions and future work are given in Sect. 5.

2

Proposed System

In this section, we present the proposed system. The structure of the proposed system is shown in Fig. 1. We implement the virtual space of the proposed system using Chai3d [24,25] which is a C++ framework for haptic, visualization and real-time interaction. For the interaction with the real and virtual space is used the Novint Falcon which is a haptic device with 3 Degrees of Freedom (3-DoF). We design the touchable device of Novint Falcon [26,27] considering the soldering iron for soldering training. In addition, we can use arbitrary Printed Wired Board (PWB) objects as soldering targets in the virtual space. The soldering iron tip in the virtual space moves along with the manipulation of the soldering iron mounted to the Novint Falcon. The proposed system supports the soldering procedure and cautions shown in Table 1 by voice output and text display on the screen. In addition, the system educates soldering based on the flowchart shown in Fig. 2. At the start of training, the soldering procedure and an explanation of the training content are displayed as shown in Fig. 3(a). During the training, the soldering procedure corresponding to the three-dimensional coordinates of the soldering iron tip is displayed as shown in Fig. 3(b). Then, in the case of correct procedure, it displays that the soldering is completed as shown in Fig. 3(c). In addition, the time for soldering completion at each point is compared with the trainer and displayed as shown in Fig. 3(d) in cases within ±10 [%]. In cases where the soldering iron tip is within the caution or danger area, the caution or warning is displayed as shown in Fig. 3(e). The caution area is the hemispherical area with a radius half of long side of the PWB shown in Fig. 4(a), excluding the soldering area. Also, the danger area is the semi-cylindrical area with a radius half of short side of the PWB shown in Fig. 4(b), excluding the soldering area. Finally, two-step feedback is performed as a result of the soldering training. In the first step, the total evaluation score is derived by adding the scores of each item based on the evaluation index x in Table 2. Then, the training results shown in Table 3 and the areas for improvement shown in Table 4 are presented to the learner as shown

56

K. Toyoshima et al.

Fig. 1. Proposed system. Table 1. Examples of support during training. State

Contents of instruction

Procedure1 • Training time is 60 [s] • The soldering procedure is A to H • Touch the soldering iron tip to the target for about 3 [s] • Please move the tip to the starting point Procedure2 • Soldering on the left side. The soldering procedure follows A, B, C and D Procedure3 • Next, right side. Moving the soldering iron tip from D to E Procedure4 • Soldering on the right side. The soldering procedure follows E, F , G and H Procedure5 • The training is finished Caution

• Be careful with the movements of soldering iron tip

Danger

• Danger!! Hight possibility of contact with PWB

in Fig. 3(f). In the second step, time series analysis is applied to the time series data of three-dimensional coordinates collected during training to present the results of dangerous movement detection. In the proposed system, we focus on the trajectory of the soldering iron tip and the moving distance in 0.1 [s] for analyzing the soldering technique. The trajectory of the soldering iron tip shows the difference in the way the trainee and the trainer use the soldering iron. From the moving distance in 0.1 [s], we can see the dangerous movements that may lead to injury or damage of the trainee while performing the soldering. Also, the moving distance shows the soldering procedure performed by the trainee. The trainee should perform

Design and Implementation of a Haptics Based Soldering Education System

57

Fig. 2. Flowchart of soldering education.

the same procedure as the trainer. Therefore, the three-dimensional coordinates of Novint Falcon grip indicating the soldering iron tip are collected at 0.1 [s] intervals during the training. In order to detect dangerous behaviors that derived during the soldering training, the proposed system detects the anomaly based on LSTM and Mahalanobis distance [28].

58

K. Toyoshima et al.

Fig. 3. Display examples for support during training.

Fig. 4. Visualization of danger areas and caution areas.

For LSTM training, the proposed system collects the training data of a reference trainer and is using the time series data converted to Euclidean distance of 0.1 [s] as relative location information. Then, the average of the movements data for the trainer collected in the same environment is used as the datasets for nor-

Design and Implementation of a Haptics Based Soldering Education System

59

Table 2. Evaluation index of soldering training. Evaluation items

Requirement

Evaluation score

Time difference

The time difference in absolute value to soldering completion is greater than 10 [%]

1

False contact

The three-dimensional coordinates of the 3 soldering iron tip are within the danger zone

Sequential mistakes Sequential mistakes have been made regarding false contacts and incorrect procedures

3

Incorrect procedure The soldering procedure is different from the trainer

3

Table 3. Four-level evaluation of soldering training. Evaluation Total score = x Excellent

x=0

Good

x=1

Bad

x=2

Terrible

x≥3

Table 4. Example of feedback on evaluation. Improvement items

Points of improvement

Time difference

The time to work from soldering target Please carefully review the A to B is different from the trainer. contact time and the movement of (The same applies to each point as A to the soldering iron tip B, B to C, etc.)

False contact

False contact detected !!

Incorrect procedure Incorrect procedure detected !!

Method of improvement

Be careful with the movement of the soldering iron tip Please double check the correct procedure

mal movements. The proposed system derives the difference vectors between the observed values and the predicted values by LSTM from the trainer and trainee data, respectively. Then, the difference vector of the trainer is matched to the normal distribution. The maximum likelihood estimator of the normal distribution is obtained by the mean vector and the variance-covariance matrix based on the maximum likelihood method. In addition, anomalies are detected by deriving the Mahalanobis distance from the mean vector and variance-covariance matrix and the difference vector of the trainee. In order to derive anomalies in soldering motion quantitatively, the proposed system labels anomalies and derives thresholds based on F -scores [29] which can be adapted to the labeled data.

60

K. Toyoshima et al.

Fig. 5. Experimental environment.

Fig. 6. Visualization of soldering procedure.

3

Experimental Scenario

In this section, we explain the experimental scenario. In Fig. 5 is shown the experimental environment and Fig. 6 the soldering procedure. The trainee performs the motion of touching the soldering iron tip to one of the soldering targets on the printed wired board for about 3 [s]. In the soldering training, the training starts when the soldering iron tip is at the starting point. The trainee performs soldering in the order from soldering target A to H. Then, the training ends when the soldering iron tip is at the end point. In addition, the trainer represents the evaluation criteria and the trainee represents the evaluation target. The trainee

Design and Implementation of a Haptics Based Soldering Education System

61

Fig. 7. Visualization results of the soldering iron tip trajectory.

Fig. 8. Visualization results of moving distance.

manipulates the touchable device to move the soldering iron tip in the virtual space. The experiment time is 60 [s]. The proposed system predicts 600 points obtained in the experimental results using LSTM and detects the anomaly for the distance in 0.1 [s] interval. In order to detect anomalies based on LSTM, we consider the deliberately of large movements and different soldering procedures of trainee training data.

62

K. Toyoshima et al.

Fig. 9. Visualization results of anomaly detection.

4

Experimental Results

In this section, we describe the experimental results. The experimental results are shown in Fig. 7, Fig. 8 and Fig. 9. Figure 7 shows the visualization results of the soldering iron tip trajectory by the trainer and trainee. It can be seen that the trainer moves the soldering iron tip less than the trainee and the soldering procedure of the trainee is different from the trainer. In Fig. 8 is shown the visualization of the distance moved of the soldering iron tip in 0.1 [s] by the trainer and trainee. The difference of about 5 [s] can be observed in the time to complete all soldering. It can be seen that the moving distance of the soldering iron tip by the trainee compared to the trainer has a large movement between 13 [s] to 14 [s] and 22 [s] to 23 [s]. In Fig. 9(a) and Fig. 9(b) are shown the anomalies associated with the distance moved between 0.1 [s] of the training performed by the trainer and trainee, respectively. The results of the trainee show that anomalies exceeding the threshold are detected at about 14 [s] and 23 [s]. The movement at about 14 [s] is the large movement performed intentionally when the soldering iron tip is touching the soldering target. Also, the soldering procedure from D to H at about 27 [s] by the trainee is different from the procedure from D to E by the trainer. On the other hand, the results by the trainer show that no anomalies exceeding the threshold in moved distance are detected. From the experimental results, we conclude that the LSTM based anomaly detection is effective method for the large movements and soldering procedures of the trainees.

5

Conclusions

In this paper, we proposed a soldering education system based on haptics. Also, we presented the experimental results of detecting anomalies with trainee data considering large movements and different procedures. From the experimental results, we conclude as following.

Design and Implementation of a Haptics Based Soldering Education System

63

• The proposed system can perform virtual training based on haptics. • The experimental results shows that the system can analyze soldering motions and detect dangerous movements. • The proposed system can safely support the learning of soldering techniques for beginners. In the future, we would like to improve the soldering education system based on the evaluation of hand blurring during training and automation supplies by considering different scenarios. Acknowledgement. This work was supported by JSPS KAKENHI Grant Number JP20K19793.

References 1. Joseph, F.J., et al.: Neurosurgical simulator for training aneurysm microsurgery-a user suitability study involving neurosurgeons and residents. Acta Neurochirurgica 162(10), 2313–2321 (2020) 2. Chen, X., Sun, P., Liao, D.: A patient-specific haptic drilling simulator based on virtual reality for dental implant surgery. Int. J. Comput. Assist. Radiol. Surg. 13(11), 1861–1870 (2018). https://doi.org/10.1007/s11548-018-1845-0 3. Bortone, I., et al.: Wearable haptics and immersive virtual reality rehabilitation training in children with neuromotor impairments. IEEE Trans. Neural Syst. Rehabil. Eng. 26(7), 1469–1478 (2018) 4. Honda, T., et al.: Ball-type haptic interface to enjoy sports games. Int. AsiaHaptics Conf. 535, 284–286 (2018) 5. Khosravi, H., et al.: Simulating mass in virtual reality using physically-based handobject interactions with vibration feedback. In: Proceedings of Graphics Interface, pp. 241–248 (2021) 6. Liu, L., et al.: Haptic technology and its application in education and learning. In: The 10th International Conference on Ubi-media Computing and Workshops (Ubi-Media), pp. 1–6 (2017) 7. Salazar, S., et al.: Altering the stiffness, friction, and shape perception of tangible objects in virtual reality using wearable haptics. IEEE Trans. Haptics 13(1), 167– 174 (2020) 8. Oda, T., et al.: Design and Implementation of an IoT-based E-learning Testbed. Int. J. Web Grid Serv. 13(2), 228–241 (2017) 9. Hirota, Y., et al.: Proposal and experimental results of a DNN based real-time recognition method for Ohsone style fingerspelling in static characters environment. In: Proceedings of IEEE GCCE-2020, pp. 476–477 (2020) 10. Ji, Y., et al.: A method for LSTM-based trajectory modeling and abnormal trajectory detection. IEEE Access 8, 104063–104073 (2020) 11. Toyoshima, K., et al.: Proposal of a haptics and LSTM based soldering motion analysis system. In: Proceedings of IEEE GCCE-2021, pp. 774–775 (2021) 12. Toyoshima, K., et al.: Proposal and evaluation of a soldering motion analysis system based on haptics. In: Proceedings of the 10th EIDWT-2022, pp. 308–315 (2022) 13. Sherstinsky, A., et al.: Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) network. Physica D: Nonlinear Phenom. 404, 1–43 (2020)

64

K. Toyoshima et al.

14. Hochreiter, S., et al.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997) 15. Hirota, Y., et al.: Proposal and experimental results of an ambient intelligence for training on soldering iron holding. In: Proceedings of BWCCA-2020, pp. 444–453 (2020) 16. Ishitaki, T., et al.: Application of deep recurrent neural networks for prediction of user behavior in Tor networks. In: Proceedings of IEEE AINA-2017, pp. 238–243 (2017) 17. Ishitaki, T., et al.: A neural network based user identification for Tor networks: data analysis using Friedman test. In: Proceedings of IEEE AINA-2016, pp. 7–13 (2016) 18. Oda, T., et al.: A neural network based user identification for Tor networks: comparison analysis of activation function using Friedman test. In: CISIS-2016, pp. 477–483 (2016) 19. Yao, L., et al.: An improved LSTM structure for natural language processing. In: The IEEE International Conference of Safety Produce Informatization (IICSPI), pp. 565–569 (2018) 20. Nagai, Y., et al.: Approach of a Word2Vec based tourist spot collection method considering COVID-19. In: BWCCA-2020, pp. 67–75 (2020) 21. Nagai, Y., et al.: Approach of an emotion words analysis method related COVID-19 for Twitter. In: IEEE GCCE-2021, pp. 1–2 (2021) 22. Nagai, Y., et al.: Approach of a Japanese co-occurrence words collection method for construction of linked open data for COVID-19. In: IEEE GCCE-2020, pp. 478–479 (2020) 23. Wang, Q., et al.: Speaker diarization with LSTM. In: The IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5239–5243 (2018) 24. Rodr´ıguez-Vila, B., et al.: A low-cost pedagogical environment for training on technologies for image-guided robotic surgery. In: Lhotska, L., Sukupova, L., Lackovi´c, I., Ibbott, G.S. (eds.) World Congress on Medical Physics and Biomedical Engineering 2018. IP, vol. 68/2, pp. 821–824. Springer, Singapore (2019). https://doi. org/10.1007/978-981-10-9038-7 151 25. Battagli, E., et al.: TcHand: visualizing hands in CHAI3D. In: The IEEE World Haptics Conference (WHC), p. 354 (2021) 26. Jose, J., et al.: Design of a bi-manual haptic interface for skill acquisition in surface mount device soldering. Solder. Surf. Mount Technol. 31(2), 133–142 (2019) 27. Ivanov, V., Strelkov, S., Klygach, A., Arseniev, D.: Medical training simulation in virtual reality. In: Voinov, N., Schreck, T., Khan, S. (eds.) Proceedings of International Scientific Conference on Telecommunications, Computing and Control. SIST, vol. 220, pp. 177–184. Springer, Singapore (2021). https://doi.org/10.1007/ 978-981-33-6632-9 15 28. McLachlan, G.: The Mahalanobis distance. Chemom. Intell. Lab. Syst. 50(1), 1–18 (2000) 29. Malhotra, P., et al.: Long short term memory networks for anomaly detection in time series. Eur. Symp. Artif. Neural Netw. 23(56), 89–94 (2015)

A Secure Data Sharing Scheme Based on CP-ABE in VANETs Xinyang Deng1 , Tianhan Gao1(B) , Nan Guo2 , and Kang Xie3 1

3

Software College, Northeastern University, Shenyang, China [email protected], [email protected] 2 Computer Science and Engineering College, Northeastern University, Shenyang, China [email protected] The Third Research Institute of Ministry of Public Security, Shanghai, China [email protected]

Abstract. Vehicle ad-hoc networks (VANETs), as the special mobile ad hoc network (MANET), are able to realize the rapid interconnection among vehicles and roadside infrastructures, so as to guarantee that vehicles obtain continuous and stable network services. As one of the vital services supported by VANETs, data sharing service supports vehicles to gain interested data information (e.g. road condition information) in time and improve the driver’s experience. However, due to the wireless network communication environment, it is easy for adversaries to forge sharing data to confuse the surrounding vehicles, or invade the privacy of the drivers by collecting the data shared from the surrounding vehicles, which may mislead drivers or even cause serious traffic accidents. This paper proposes a secure data sharing scheme in VANETs. By integrating ciphertext policy attribute-based encryption mechanism (CP-ABE), the privacy of the data sender is protected and the data receiver is able to obtain reliable sharing data in time. Analysis results show that the security and performance of the scheme are improved compared with the traditional ones.

1

Introduction

With the development of intelligent transportation and smart city, vehicular ad-hoc networks (VANETs) are proposed and show the advantages for vehicles to provide network communication [1]. However, due to the complex wireless network environment and dynamic topology, VANETs have to face the technical issues of how to protect the vehicles’ privacy and provide stable network services for vehicles [2]. Recently, IEEE proposed the standard for Wireless Access in Vehicular Environments Architecture [3] and gave the suggestions to solve the above issues. Now, VANETs have supported a variety of services related to safety, efficiency, and entertainment for vehicles. As the basic service of traffic control, collision warning, and other safety services, data sharing is becoming very important in the service system of VANETs. c The Author(s), under exclusive license to Springer Nature Switzerland AG 2022  L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 65–74, 2022. https://doi.org/10.1007/978-3-031-08819-3_7

66

X. Deng et al.

Recently, a series of data sharing schemes for VANETs is proposed. [4] proposes a revocable data-sharing scheme (IRD), this scheme adopts ciphertextpolicy attribute-based encryption (CP-ABE) to support VANETs data-sharing. In addition, cloud computation is employed to alleviate the computation cost of resource-constrained devices. However, since a large amount of data from vehicles are collected and shared by single authority, once the authority is attacked by the internal adversaries, all data is at risk of being exposed. [5] and [6] employ multi-authorities storage and data access control strategy to solve the above problem respectively. [5] constructs a secure decentralized access control scheme (MAEAC). Based on multi-authorities CP-ABE, MAEAC supports data privacy preservation and access control, the private of stored data cannot be resolved by any single authority. [6] adopts secure k-nearest neighbor algorithm (KNN) and polynomial fitting to achieve the goal of precise data search. Meanwhile, the proposed scheme refines users’ rights based on spatial data access control strategy to ensure the security of data stored. However, since all data is collected and transmitted by cloud, the real-time data cannot be shared to vehicles in VANETs in time due to high transmission delay. [7] adopts pull and push-based mechanism to support vehicles disseminate necessary data. In the proposed scheme, the vehicle holding the data is able to find the location of receivers by beacon, and dynamically adjust the area where the vehicle transmits the data according to the needs, so that receivers can obtain necessary data in time. Different from the traditional data distribution scheme based on fixed area, dynamically adjusting the data distribution area can improve the success rate of data requests effectively. Nevertheless, this scheme does not give the details how to guarantee the security and reliability of the data. Once the data comes from the adversaries, the receiver may cannot obtain the correct information, which may confuse drivers. [8] proposes a verifiable data sharing scheme based on blockchain. The vehicles holding data are able to define the access policy and support one-to-many data sharing. Depending on blockchain, the sender cannot deny the sent data and the data in the cloud cannot be tampered with, which guarantee data confidentiality and reliability. However, the low storage and query efficiency of the blockchain make it difficult for receivers to obtain the data in time. In order to solve the above issues, this paper proposes a secure data sharing scheme based on CP-ABE in VANETs. The contributions are summarized as follows. • RSU is responsible for data collection and sharing to solve the problems of high cloud transmission delay and the low blockchain storage and query efficiency. • Based on CP-ABE mechanism, the sender is able to determine the access rights of receiver. Thus, only the receiver who meets the attributes confirmed by the sender can obtain the data. In addition, RSU cannot tamper with data from the sender and the receiver is able to receive reliable data in time without mutual authentication with RSU. • Security and performance analysis show that the proposed scheme is secure and effective.

A Secure Data Sharing Scheme Based on CP-ABE in VANETs

67

The remainder of the paper is organized as follows. Section 2 introduces the preliminaries. Section 3 gives the details of the proposed scheme. Section 4 discusses the result of the proposed scheme in security and performance. Finally, Sect. 5 concludes the paper and presents the future work.

2 2.1

Preliminaries VANETs

Vehicular ad-hoc networks (VANETs), as the novel network providing communication services for vehicles, have become a hot spot in academic research. In VANETs, dedicated short range communication technology (DSRC) [9] is adopted to guarantee the stable data transmission in the dynamic network topology, which makes it possible for the data sharing [3]. Now, VANETs not only support to transmit the real-time information to help traffic control centers to take actions to maintain traffic safety, but also effectively assist vehicles to obtain surrounding traffic status information quickly. VANETs consist of vehicles and roadside infrastructure (e.g. Roadside unit, RSU). RSU is responsible for providing network communication services for surrounding vehicles and supports to provide driving and road condition warning services. Vehicles are able to obtain a variety of application services through the communication with other vehicles and RSUs. In addition, each vehicle deployed with GPS can get accurate location information and know the driving status of nearby vehicles through its own sensors and beacon messages received from other vehicles. 2.2

Bilinear Pairing and BDHE Assumption

Definition 1 (Bilinear pairing). Let G1 and GT be the additive cyclic group and the multiplicative cyclic group of prime order q with λ bits respectively. e : G1 × G1 → GT , as the bilinear pairing, is requested to meet the following properties. • Bilinearity: For all P, Q, ← G1 and a, b ← Zq∗ , there is e(aP, bQ) = e(P, Q)ab . • Non-degeneracy: Exist P, Q ← G1 , e(P, Q) = 1GT . • Computability: For all P, Q ← G1 , there exists an efficient algorithm to calculate e(P, Q). Definition 2 (BDH assumption). Let G1 be the additive cyclic group with large prime order p. P is the generator of G1 and λ is security parameter. Given a (2n+1)-tuple (h, P , αP , ..., αn P , αn+2 P , ..., α2n P ) ∈ G1 , there exists no PPT n+1 algorithm A that can output e(P, h)α with negligible probability negl(λ).

68

2.3

X. Deng et al.

Ciphertext Policy Attribute-Based Encryption

The first CP-ABE scheme was proposed by [10], where access structures are embedded into the ciphertext and the key is associated with user’s attributes. The user’s attributes used for key generation are requested to satisfy the access policy. Consequently, the data owner is able to determine which user can decrypt ciphertext. A CP-ABE scheme includes a tuple of four algorithms: Setup, KeyGeneration, Encryption, and Decryption. • (param, P K, M SK) ← Setup(1λ , k). Input security parameter λ and the number of attributes in the system k, the Setup algorithm outputs system parameters param, public key P K, and master key M SK. • SK ← KeyGeneration(param, P K, List). The KeyGeneration algorithm takes input of param, P K, and user’s attribute list List and returns private key of users SK. • C ← Encryption(param, P K, W, M ). Taking param, P K, specified access policy W , and message M as input, the Encryption algorithm returns ciphertext C associating the anonymized access policy W  . • M ← Decryption(param, P K, SK, C). The Decryption algorithm takes param, P K, SK, and C as input and returns valid message M if List meets W. The proposed scheme adopts Zhou’s scheme [11] to support data sharing due to its high security and efficiency.

3

The Proposed Scheme

This section gives the details of the proposed scheme, which include network architecture, system initialization, vehicle registration protocol, and data sharing protocol. The symbols and descriptions are depicted in Table 1. Table 1. Symbols and descriptions Symbol

Description

IDA

Entity A’s real identity

P KA

The public key of entity A

SKA

The private key of entity A

KA−B

The session key between entity A and entity B

param

The system parameters

ListA

Entity A’ s attribute list

W

Access policy

Ai

The i-th attribute

Enc K{M } Encrypt message M with the key K CA−B

The ciphertext generated by entity A and sent to entity B

A Secure Data Sharing Scheme Based on CP-ABE in VANETs

3.1

69

Network Architecture

As shown in Fig. 1, the network architecture of the proposed scheme includes trusted authority (TA), RSUs, and vehicles. (1) TA is responsible for generating and broadcasting system parameters and issuing public/private key and related pseudonym certificates for registered vehicles and RSUs to support mutual authentication and data sharing service. (2) RSUs are able to collect the data of nearby vehicles holding data through V2I communication after finishing mutual authentication, and timely share these data to other vehicles. (3) The vehicles, as the data owners, can send data to the trusted RSUs and determine the data access policy through CP-ABE mechanism. Meanwhile, The vehicles who meet the policy, as the receivers, can get the stored data in RSUs. 3.2

System Initialization

In system initialization, TA generates and broadcasting system parameters to support data sharing service for vehicles. The details are shown as follows. (1) TA selects an additive group G1 and a multiplication group GT respectively, where the prime order is q and the generator is P . Besides, a bilinear pairing e : G1 × G1 → GT is selected.

Fig. 1. VANETs architecture.

70

X. Deng et al.

(2) n attributes {A1 , A2 , ...An } are defined and each attribute owns three value − + − ∗ {A+ i , Ai , Ai }, where Ai expresses owning Ai attribute, Ai means no Ai ∗ attribute, and Ai refers to do-not-care. The mapping between the attribute values and integer numbers are depicted in Table 2. (3) TA chooses its master keys α, β ∈ Zq∗ and computes P KT A = {H, P K1 , P K2 , ..., P KN , P KN +2 , P KN +3 , ..., P K2N }, where P Ki = (α)i ·P , N = 3n, H = β · P. (4) TA broadcasts the system parameters param = {G1 , GT , q, e, P, P KT A } and stores master keys α, β locally.

Table 2. Mapping attribute values to numbers

3.3

Attributes A1

A2

A+ i A− i A∗i

A3

... An

1

2

3

... n

n+1

n+2

n+3

... 2n

2n+1 2n+2 2n+3 ... 3n

Vehicle Registration Protocol

In this section, each legal vehicle is able to send its real identity to TA for registration. The details are shown in Fig. 2.

Fig. 2. Vehicle registration protocol.

A Secure Data Sharing Scheme Based on CP-ABE in VANETs

71

(1) Vehicle v chooses random number t ∈ Zq∗ and encrypts its true identity IDv ∈ G1 to get ciphertext Cv−T A = Enc H{IDv } = {V1 , V2 }, where V1 = t · P and V2 = IDv + t · H, t ∈ Zq∗ . (2) Vehicle v sends Cv−T A to TA. (3) TA decrypts Cv−T A by computing IDv = V2 − βV1 . For the vehicle tagged with the attribute list Listv = {listv [i]i∈[1,n] | list vn[i] ∈ [1, 2n]}, TA selects n random numbers {ri }i∈[1,n] ∈ Zq∗ and sets r = i=1 ri . Then, TA computes SKv = {T, {Ti }i∈[1,n] , {Fi }i∈[1,n] }, where T = r · h, Ti = (αlistv [i] + ri ) · h, Fi = (α2k+i +ri )·h. Finally TA computes the session key KT A−v = β·V1 and uses AES mechanism to encrypt SKv to get CT A−v = Enc KT A−v {SKv }. (4) TA sends CT A−v to vehicle. (5) When receiving CT A−v , vehicle first sets the session key Kv−T A = t · H and decrypts the ciphertext CT A−v to get SKv . 3.4

Data Sharing Protocol

Vehicle can send data to trusted RSU and support other vehicles meeting access policy to obtain the data shared. The details are shown as follows. (1) Vehicle vi holding data M and the access policy Wvi with n attributes chooses random d ∈ Zq∗ and computes symmetric encryption key K = e(P KN , P K1 )n·d . (2) Vehicles vi adopts AES mechanism and uses K to encrypt shared data M 0 1 to get C vi = Enc K{M }. In addition, vi computes Cvi = d · P , Cvi = d · (h + i∈Wv P KN +1−i ). vi anonymizes the access policy by removing all i  attributes values except A∗i and sets Wv i = Wvi {A∗ }i∈[1,n] . Consequently, i vi gets ciphertext of M : {Wv i , Cvi , Cv0i , Cv1i }. (3) Vehicles vi sends Wv i , Cvi , Cv0i , Cv1i to RSU via secure channel. (4) RSU stores the ciphertext and identity idinf ovi from vi locally. (5) If the vehicle vj within the RSU communication range applies to RSU for interested data, vj constructs a local guess of access policy Wvj and replies for the data from vi via RSU. (6) When receiving W  , Cvi , Cv0i , Cv1i , vehicle vj constructs local guess of access policy Wvj . For i ∈ [1, n], vehicle vj computes Vi−0 = e(P KWvj [i] , Cv1i ). In addition, if Wvj [i] ∈ Listvj , vj computes Vi−1 = e(Ti +



P KN +1−j+Wvj [i] , Cv0i ).

j∈Wvj ,j=Wvj [i]

Else if W [i]vj ∈ {A∗vj }i∈[1,n] , vj computes Vi−1 = e(Fi +



P KN +1−j+Wvj [i] , Cv0i ).

j∈Wvj ,j=Wvj [i] N +1

Then, vj sets Vi−0 /Vi−1 = e(P, P )−dβri +dα

.

72

X. Deng et al.

(7) Vehicle vj sets n 

N +1

Vi−0 /Vi−1 = e(P, P )−d·β·(r1 +r2 ,...+rn )+n·d·α

i=1 N +1

= e(P, P )−d·β·r+n·d·α

and e(T, Cv0i ) = e(P, P )d·β·r . The key of ciphertext Cvi is K=(

n 

Vi−0 /Vi−1 ) · e(T, Cv0i )

i=1 N +1

= e(P, P )−d·β·r+n·d·α

· e(P, P )d·β·r

N +1

= e(P, P )n·d·α

= e(αN · P, α · P )n·d = e(P KN , P K1 )n·d . Finally, Vehicle vj uses K decrypts Cvi to get M .

4 4.1

Security and Performance Analysis Security Analysis

(1) User Privacy. In data sharing protocol, vj only gets the ciphertext W  , Cvi , Cv0i , Cv1i , any private information containing the sender (e.g. identity information, driving trajectory) cannot be obtained depending on ciphertext. (2) Confidentiality. Only the receiver that meets access policy can decrypt the ciphertext with its attribute parameters. Otherwise, the data receiver can only obtain the random string through the ciphertext. (3) Non-tamper: Since cannot obtain the attribute-based private key issued by TA, RSU cannot obtain or modify the content of message M by decrypting the ciphertext. (4) Unforgeability: As the private key SKvi of vehicle vi is confidential to other entities, RSU cannot forge a legal vehicle to generate the legal ciphertext of data M , and any illegally forged data can be identified by receiver. 4.2

Performance Analysis

In this section, the performance of the proposed scheme is discussed in secure data sharing protocol in terms of computational cost compared with IRD [4] and MAEAC [5]. Computational cost refers to the total computation time required for data sharing. We denote the time cost of point multiplication by Tpm , the time cost of map-to-point by Tmtp , and the time cost of bilinear pairing by Tbp . In IRD, Given message M and n attributes, data owner executes 3n+2 point multiplication operations and 1 bilinear pairing operations to get the ciphertext

A Secure Data Sharing Scheme Based on CP-ABE in VANETs

73

C. For receiver, n + 1 point multiplication operations and n + 1 bilinear pairing operations are executed to get the message M . For MAEAC, in order to encrypt message M , data owner executes n+6 point multiplication operations, n + 3 bilinear pairing operations and n map-to-point operations. For data receiver, n point multiplication operations, 2n map-to-point operations, and 3n bilinear pairing operations are executed to obtain message M. In the proposed scheme, in order to compute the ciphertext of M : {W  , Cvi , Cv0i , Cv1i }, vehicle vi holding data is request to execute 2 point multiplication operations. When getting the ciphertext from RSU, vehicle vj needs to execute n + 2 bilinear pairing operations to obtain interested message M . Table 3 shows the comparisons of the computational cost for data sharing. We can see that the proposed scheme owns low computational cost due to less operations. Table 3. The computational cost of each scheme Protocol

Data encrypt

Data decrypt

IRD

(3n + 2)Tpm + Tbp

(n + 1)Tpm + (n + 1)Tbp

MAEAC

(n + 6)Tpm + (n + 3)Tbp + nThtp nTpm + 3nTbp + 2nThtp

The proposed scheme 2Tpm

5

(n + 2)Tbp

Conclusion

In order to solve the low storage and computation efficiency of tradition schemes and protect the privacy of data owners, this paper proposes a secure data sharing scheme based on CP-ABE in VANETs. RSU supports data collection and sharing. Services. Data owner determines the access policy and the vehicles that meet the policy can obtain data securely. The analysis shows that the security and performance of the proposed scheme are improved compared with the traditional ones. In the future research, we will adopt Veins to simulate the proposed scheme and evaluate the performance in experiment. Acknowledgment. This work was supported by National Natural Science Foundation of China under Grant Number 52130403, Fundamental Research Funds for the Central Universities under Grant Number N2017003, and The Key Lab of Information Network Security, Ministry of Public Security under Grant Number C20600.

References 1. Mchergui, A., Moulahi, T., Zeadally, S.: Survey on Artificial Intelligence (AI) techniques for Vehicular Ad-hoc Networks (VANETs). Veh. Commun. 34, 100403 (2022)

74

X. Deng et al.

2. Qu, F., Wu, Z., Wang, F.-Y., Cho, W.: A security and privacy review of VANETs. IEEE Trans. Intell. Transp. Syst. 16(6), 2985–2996 (2015) 3. IEEE Guide for Wireless Access in Vehicular Environments (WAVE) Architecture Redline. IEEE Std 1609.0-2019 (Revision of IEEE Std 1609.0-2013) - Redline, pp. 1–219, 10 April 2019 4. Horng, S.-J., Lu, C.-C., Zhou, W.: An identity-based and revocable data-sharing scheme in VANETs. IEEE Trans. Veh. Technol. 69(12), 15933–15946 (2020) 5. Hong, Z., et al.: Multi-Authority attribute-based encryption access control scheme with policy hidden for cloud storage. Soft. Comput. 22(1), 1–9 (2016) 6. Xu, G., Li, H., Dai, Y., Yang, K., Lin, X.: Enabling efficient and geometric range query with access control over encrypted spatial data. IEEE Trans. Inf. Forensics Secur. 14(4), 870–885 (2019) 7. Nakamura, N., Niimi, Y., Ishihara, S.: Live VANET CDN: adaptive data dissemination scheme for location-dependent data in VANETs. In: 2013 IEEE Vehicular Networking Conference, pp. 95–102 (2013) 8. Fan, K., et al.: A secure and verifiable data sharing scheme based on blockchain in vehicular social networks. IEEE Trans. Veh. Technol. 69(6), 5826–5835 (2020) 9. Bera, R., Bera, J., Sil, S., Dogra, S., Sinha, N.B., Mondal, D.: Dedicated short range communications (DSRC) for intelligent transport system. In: 2006 IFIP International Conference on Wireless and Optical Communications Networks, p. 5 (2006) 10. Bethencourt, J., Sahai, A., Waters, B.: Ciphertext-policy attribute-based encryption. In: IEEE Symposium on Security and Privacy 2008, pp. 321–334 (2007) 11. Zhou, Z., Huang, D., Wang, Z.: Efficient privacy-preserving ciphertext-policy attribute based-encryption and broadcast encryption. IEEE Trans. Comput. 64(1), 126–138 (2015)

An Anonymous Authentication Scheme Based on Self-generated Pseudonym for VANETs Jiayu Qi and Tianhan Gao(B) Northeastern University, Shenyang, Liaoning, China [email protected]

Abstract. With the maturity of mobile communication and Internet of Things, the extensive deployment of Vehicular Ad-hoc Networks (VANETs) and the popularity of related applications have gradually become the research hotspot. Vehicles can access VANETs to obtain better travel choices, more third-party services, and share social and traffic information with other vehicles. Significantly, security and privacy protection is a major issue in VANETs. As the first line of defense against the attacks, anonymous authentication is a necessary approach to preserve user privacy. Whereas, the existing anonymous authentication schemes cannot maintain a balance between security and efficiency. In this paper, we propose an anonymous authentication scheme based on self-generated pseudonym in VANETs. The proposed scheme avoids the overhead of certificate management through the self-generation of pseudonym and provides conditional privacy protection. The security and performance analysis shows that the proposed scheme effectively guarantees the security and anonymity of vehicles and owns high efficiency.

1

Introduction

With the continuous development of Vehicle Ad-hoc Networks (VANETs) [1], the level of Intelligent Transportation Systems (ITSs) [2] are improved so that the vehicle network services and applications are more attractive to the public. VANETs consist of vehicles equipped with on-board units (OBUs), road-side units (RSUs) and a trusted authority (TA). Two categories of communications are possible: vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I). The OBU enables the vehicle to communicate with other entities. The hybrid communication mode is also called V2X communication [3]. With the use of VANETs, higher road safety and better driving experiences can be achieved in a variety of promising applications. For example, vehicles equipped with sensors and Global Positioning System (GPS) equipment can monitor road conditions, report the detected traffic information to local authorities through V2V and V2I communication, and issue alerts in the region to remind other vehicles to drive carefully. Although VANETs can benefit users through abundant and various applications, c The Author(s), under exclusive license to Springer Nature Switzerland AG 2022  L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 75–84, 2022. https://doi.org/10.1007/978-3-031-08819-3_8

76

J. Qi and T. Gao

it is precisely because of the open and hybrid architecture that allows information and important data in the network to be monitored in real time. Malicious attackers can intercept messages of interest to them and attempt to tamper with or forge messages to abuse VANETs. Therefore, a powerful authentication and message integrity mechanism is the key to the security of VANETs [4,5]. In particular, location privacy is a basic parameter to determine the privacy protection of VANETs. Based on the use scenario of VANETs, the location of the vehicle is closely related to the user driving the vehicle. Any private information such as location privacy being disclosed will lead to the personal privacy information of the vehicle user not being protected. In addition, if the authentication mechanism in VANETs is flawed, malicious vehicles [6] may pretend to be legal ones to abuse and destroy VANETs or obtain illegal benefits [7]. In order to make up for the above defects, one of the recommended methods in VANETs is that each vehicle regularly changes the identifier(also known as a pseudonym) when broadcasting security messages, and generates a signature associated with the pseudonym for authentication instead of using only one static identifier. In general, a set of pseudonyms is stored in the OBU and acts as a usable identifier, greatly enhancing the privacy of the user’s identity. At the same time, since vehicles use different pseudonyms on the road, the unlinkability of pseudonyms can ensure the location privacy of vehicles. Such pseudonym-based schemes [8–10] at present, however, have been proved to be unable to satisfy non-repudiation. While, another popular approach is identity-based signature scheme [11–13], in which the user’s public key is the node’s identifier itself. The private key can be generated by the authorized entity from the identifier. Although this scheme also avoids certificate management, key escrow has become a new challenge. In addition, there have been some researches on group signature [14–16] in recent years, which enables all group members to generate signatures using their private keys as the whole group and the signatures can be verified by the group public key. Group signature schemes avoid certificate management, but the cost is huge group management overhead. This paper presents a self-generated pseudonym based anonymous authentication scheme (SGAAS) for VANETs. On account of the non-interactive zeroknowledge proof mechanism, the pseudonym-based anonymous authentication protocol is implemented, where the offline update of pseudonyms is supported. The vehicle can calculate new pseudonyms locally without TA’ participation in order to avoid using pseudonym certificate. Theoretical analysis and performance comparison show that the proposed scheme owns better performance while solving the defects of other schemes at present.

2 2.1

Preliminaries Bilinear Pairing

Let G1 be additive cycle group, the prime order is q, and GT be multiplicative group of the same order. A bilinear pairing e: G1 × G1 → GT satisfies the following properties.

An Anonymous Authentication Scheme

77

(1) Bilinear: For any P, Q ∈ G1 , a, b ∈ Zq∗ , there are e(aP, bQ) = e(P, Q)ab . (2) Non-degeneracy: Existing a certain P, Q ∈ G1 satisfies e(P, Q) = 1. (3) Computability: An efficient algorithm can calculate e(P, Q) ∈ GT , where P, Q ∈ G1 . 2.2

Mathematical Hard Problems

(1) Let G1 be additive cycle group of prime order p and P be a generator of G1 . The q-strong Diffie-Hellman problem(q-SDHP) is defined as follows: Given 1 P ) ∈ Zp × G1 . (P, xP, x2 P, ..., xq P ) for x, q, c ∈ Zp∗ to compute (c, x+c (2) Let G1 , GT and e be defined in Sect. 2.1. Let k, n be integers, P, P1 , . . . , Pn ∈ 1 P | 1 ≤ i ≤ k, 1 ≤ G1 and x, a1 , . . . , ak ∈ Zq∗ . Given {P, xP, Pj , xPj , ai , x+a i 1 / {ai Pj | 1 ≤ i ≤ k, 1 ≤ j ≤ n}. j ≤ n}, to compute x+a P for some aPj ∈ 2.3

Fiat-Shamir Heuristic

The Fiat-Shamir heuristic [17] outputs a non-interactive zero-knowledge proof(ZKP) in the random oracle model. The idea is that the Fiat-Shamir transformation takes an interactive public input and replaces the challenges with the output of a cryptographic hash function. Based on this, Fiat-Shamir heuristics can be used to construct digital signature schemes based on non-interactive knowledge proof. In interactive proof, there is an interaction between prover P R and verifier V E. P sends a ”commitment” T to V E, V E returns a random nonce to P R, and then P R sends the ”answer” to V E for verification. The Fiat-Shamir heuristic replaces step 2 with a hash function, where P R directly calculates a hash function as a random number. Therefore, in a non-interactive certification, only one message sent by P R to V E contains a certification that can be regarded as a digital signature. The proof generated according to the following steps: (1) P R holds a secret a, which is the discrete logarithm of A = aP , where P is a generator of a cycle group G1 with prime order q. To prove that he/she knows a, P R selects t ∈ Zq∗ randomly and calculates T = tP . (2) P R generates c = h(P, A, T ), where h() is a hash function. P R computes r = t − ac(mod p). (3) P R sends (A, T, r) as the proof to V E.  (4) After receiving (A, T, r), V E calculates c = h(P, A, T ) and computes T =  rP + cA to judge whether T = T . If the equation holds, the verification is proved to be successful.

3

The Proposed Scheme

The proposed anonymous authentication scheme adopts a pseudonym-based signature scheme which is constructed from Fiat-Shamir heuristic to achieve authentication and conditional privacy protection in VANETs. The scheme is composed of four phrases: setup, V2I authentication protocol, V2V authentication protocol, and pseudonym fast updating and tracking mechanism.

78

3.1

J. Qi and T. Gao

Setup

In this phase, TA prepares all necessary public and private parameters, and provides the registration service to OBU and RSU. 3.1.1 System Initialization During the system initialization, TA needs to process the following steps. (1) TA generates a cyclic additive group G1 and a cyclic multiplicative group GT with the same prime order q. e : G1 × G1 → GT is a bilinear pairing. (2) TA chooses a generator P of G1 and picks a random number s ∈ Zq∗ as the master secret key. Then, TA calculates the system public key Ppub = sP , and sets g = e(P, P ). (3) TA defines hash functions H1 and H2 , where H1 , H2 : {0, 1}∗ → Zq∗ . TA specifies a symmetric encryption algorithm Enc() for secure channels and chooses a secure and efficient ID-based signature for RSU. (4) TA randomly selects several t1 , t2 , t3 , ..., tn ∈ G1 and computes Ti = sti , where i = 1, 2, 3, ..., n. TA periodically updates the values of t1 , t2 , t3 , ..., tn and T1 , T2 , T3 , ..., Tn in order to allow vehicles in the system to generate new pseudonyms. (5) TA publishes the public system parameters: param = (G1 , GT , q, e, P, Ppub , g, H1 , H2 ). 3.1.2

OBU Registration Protocol

(1) OBU sends its real ID to TA through the secure channel. (2) TA generates an SDH pair (μ, Su ) as the credential Cred for the OBU. TA takes out the value of t1 , t2 , t3 , ..., tn and T1 , T2 , T3 , ..., Tn in the current time period, and generates signatures δi on (ti , Ti ). In addition, TA calculates initial pseudonyms ps1 , ps2 , ps3 , ..., psn of the OBU in advance, that is, psi = μti , and then records their mapping relationship with the OBU in the local database. When TA updates t1 , t2 , t3 , ..., tn and T1 , T2 , T3 , ..., Tn , the values recorded in the database will be updated accordingly. TA sends (t1 , T1 , δ1 ), ..., (tn , Tn , δn ) to the OBU as tokens, along with the credential Cred. (3) After obtaining tokens and Cred, OBU can easily verify the validity of Cred by verifying whether the equation e(μP, Su ) = g holds, and the validity of each token is verified by signature δ. (4) OBU stores (t1 , T1 , δ1 ), ..., (tn , Tn , δn ) and Cred. 3.1.3

RSU Registration Protocol

(1) RSU sends its serial number RID to TA through the secure channel. (2) TA generates the private key skR = H1 (R1ID )+s P for the RSU. The corresponding public key pkR is the RID of RSU. After that, TA replies skR to RSU through the secure channel. (3) RSU stores the received skR .

An Anonymous Authentication Scheme

3.2

79

V2I Authentication Protocol

When the OBU moves to the wireless communication range of the accessible RSU, the V2I authentication protocol between the OBU and the RSU will be executed. The OBU has received param published by TA. First, the OBU randomly picks a token (ti , Ti , δi ) and prepares the message m, which is generated for Elliptic Curve Diffie-Hellman key negotiation in order to establish a secure channel with the RSU. Then the OBU generates a random number Rndv ∈ Zq∗ , and calculates m = Rndv · P . The specific process of V2I authentication is as follows: (1) The RSU periodically broadcasts beacon which contains RID to all the OBUs in the communication range. (2) When the OBU receives the beacon with RID , it indicates that the OBU has entered the management area of the RSU. V2I authentication is triggered immediately. The OBU computes its temporary pseudonym psi = μti . Then the OBU chooses random numbers α, β, γ ∈ Zq∗ , computes U = αSu , and calculates V = βti and ξ = e(ti , P )γ as the commitments of psi and U , respectively. After that the OBU computes challenge c = H2 (m  psi  U  V  ξ  T S1 ), where T S1 is the timestamp. Then the OBU computes z1 = γ − cα and z2 = β − cμ. The signature σ1 on m is (U, c, z1 , z2 ). The OBU generates the authentication message ((ti , Ti , δi ), psi , m, σ1 , T S1 ) sent to the RSU. (3) After receiving the message from the OBU, the RSU checks the timestamp T S1 . If T S1 is fresh, the RSU verifies the validity of (ti , Ti ) according to δi by using the system public key Ppub . If above verifications have passed, the RSU computes ξˆ = e(ti , P )z1 · e(psi + Ti , U )c and Vˆ = z2 ti + cpsi . Then calculates cˆ = H2 (m  psi  U  Vˆ  ξˆ  T S1 ). The RSU checks whether cˆ = c, if the equation is satisfied, the OBU is regarded as legitimate. The RSU accepts m and chooses a random Rndr ∈ Zq∗ . The shared key Kv−r = Rndr · m is figured out. The RSU calculates θ = Rndr · P and generates the signature σ2 on (θ, T S2 ). The RSU replies θ, σ2 and T S2 to the OBU. (4) When the OBU receives θ, σ2 and T S2 , it first check the freshness of T S2 . If T S2 is fresh, the OBU verifies σ2 by using RID . After the verification is successful, the shared key Kv−r = Rndv · θ can be easily obtained so that a secure channel with the OBU and the RSU is established. The secure channel will be used in the subsequent pseudonym updating. 3.3

V2V Authentication Protocol

When the vehicle needs to communicate with other vehicles such as exchanging security messages, V2V authentication is performed. Assuming that OBUj receives ((ti , Ti , δi ), psi , mi , σ3 , T S3 ) from OBUi where mi contains security message, V2V authentication protocol is executed as follows: (1) OBUj checks T S3 and verifies the validity of (ti , Ti ). If T S3 is fresh and δi is a valid signature on (ti , Ti ), OBUj computes ξˆ = e(ti , P )z1 · e(psi + Ti , U )c

80

J. Qi and T. Gao

and Vˆ = z2 ti + cpsi . Then calculates cˆ = H2 (mi  psi  U  Vˆ  ξˆ  T S3 ). OBUj checks whether cˆ = c, if the equation holds, OBUj accepts mi and prepares its security message mj and the corresponding signature. OBUj randomly selects a token (tj , Tj , δj ) and generates the authentication message ((tj , Tj , δj ), psj , mj , σ4 , T S4 ). (2) OBUj returns ((tj , Tj , δj ), psj , mj , σ4 , T S4 ) to OBUi . (3) When OBUi receives ((tj , Tj , δj ), psj , mj , σ4 , T S4 ) from OBUj , OBUi performs the verification procedure similar to that of OBUj to determine whether the signature σ4 is legitimate, and OBUi accepts the security message mj if the verification is successful. 3.4

Pseudonym Fast Updating and Tracking Mechanism

In the proposed scheme, vehicles gains multiple tokens to generate multiple pseudonyms when registering with TA initially. However, since the system needs to ensure the privacy intensity required by vehicular users, the pseudonyms used by OBU should be changed regularly. TA periodically updates t1 , t2 , t3 , ..., tn and T1 , T2 , T3 , ..., Tn , and sends them to all legitimate vehicles through each RSU. In the meantime, TA calculates the new pseudonyms of all legitimate vehicles and updates the database. When the OBU receives the new values, it can generate several new pseudonyms without applying to TA or interacting with TA. The proposed SGAAS can provide a convenient, efficient and secure pseudonym updating method. When the OBU has malicious behavior in VANETs, such as spreading disloyal traffic information, all the pseudonyms(including those that have not been used yet) of the OBU should be traced and revoked. Specifically, the process of pseudonym resolution and revocation is described in detail as follows. (1) When a vehicle OBUj receives the message M sent by OBUi and considers the security message mi in M to be false, OBUj forwards the message M to the RSU within the domain. (2) After receiving M , the RSU first needs to parse M and verify the signature involved to confirm the source. Then the RSU checks whether mi is a malicious message. If so, it further sends a malicious OBU report including the pseudonym psi used by OBUi to TA. (3) After receiving the report of psi , TA can trace the vehicular user by searching the real identity corresponding to the pseudonym psi in the database, and can get all the other available pseudonyms of the user at the same time.

4

Security and Performance Analysis

In this section, the performance comparison among the proposed SGAAS and literature ones is discussed.

An Anonymous Authentication Scheme

4.1

81

Security Analysis

In this paper, a pseudonym-based signature scheme is adopted to realize anonymous authentication. Through Fiat-Shamir heuristic, the ZKP of credential (μ, Su ) possession is converted into a digital signature scheme. The proposed signature scheme can be proved to be existentially unforgeable against the adaptively chosen-message attack under the random oracle model, that is, under the assumption that (k,n)-CAA problem is intractable, no adversary is able to forge a new signature with an advantage ε to win the adaptively chosen-message attack game, thus ensuring the security of the proposed anonymous authentication scheme. Meanwhile, the proposed SGAAS meets the following security and privacy requirements in VANETs: • Unforgeability: The credentials can only be issued by TA and kept locally in secret by OBUs. All OBUs are incapable of generating or forging the signatures of others. Only a legally registered OBU is capable to use the credential to compute available pseudonyms. • Anonymity: The proposed anonymous authentication achieves a method whereby systems can securely identify legitimate OBUs without exposing private information. In V2X authentication, OBU uses pseudonyms to authenticate with other entities. These pseudonyms do not contain any identifiable information and cannot be used to link to a particular OBU or to other pseudonyms of the same OBU. Only TA is able to recover the real identity of the OBU. • Unlinkability: All t1 , t2 , t3 , ..., tn and T1 , T2 , T3 , ..., Tn are randomly generated by TA. Thus, there is no correlation between all pseudonyms of OBU. Adversaries cannot perform correlation analysis on multiple pseudonyms, i.e. given psi and psi + 1 , it is computationally hard to decide that they are correspondence to the same OBU in series without knowing μ of the OBU. • Authorization and access control: TA issues credentials to legitimate OBUs registered with it as an access authorization to VANETs. Only OBUs with credentials can self-generate available pseudonyms and legitimate signatures to pass anonymous authentication with RSU and other OBUs. After completing anonymous authentication with RSU, OBU is allowed to access VANETs and can apply for and obtain various services in VANETs. • Non-repudiation and tracing: The proposed SGAAS enables traceability once a vehicle behaves improperly. TA can reveal the real identity of the malicious OBU from its pseudonym. The initial registration requires vehicles to submit identity information to TA and all credentials are acquired on a trusted and secure channel. The pseudonym self-generation requires a valid credential which can only be issued by TA. Since OBU has no way to know or forge other credentials, it cannot generate a signature in the name of another OBU. In other words, the OBU cannot deny that it generated a signature to avoid accountability.

82

4.2

J. Qi and T. Gao

Performance Analysis

To analyze and compare the computational costs of other schemes mentioned in [12,13,15], let Tbm represents the processing time of a bilinear map, Tpm the processing time of a point multiplication in G1 and Tmtp the processing time of a map-to-point hash function in G1 . We test the computational cost of each operation on a PC with Intel Core i7-6700 3.41GHz CPU and 8 GB RAM running the cryptography library JPBC. According to our experimental results, we found that Tbm , Tpm and Tmtp have a running time of 8.58 ms, 0.96 ms and 17.43 ms, respectively. Let PKS denotes operations of Pseudonym generation, Key generation, and Signing of a message, VMS denotes operations of Verification of Message Signature, and the time needed to perform V2X authentication denoted as SUM. We draw the conclusions in Table 1. It shows that the proposed SGAAS is better than [12,15] in computational efficiency and next only to [13]. As for the communication cost, the size of each element in G1 is 128 bytes and the size for general hash function: {0, 1}∗ → Zq∗ and timestamp T S are considered to be 20 bytes and 4 bytes, respectively. As Table 2 shows, the proposed SGAAS has lower communication cost than other systems [12,13,15]. Table 1. Comparison of computational costs Scheme PKS

VMS

[12]

5Tpm + 2Tmtp = 39.66 ms

3Tbm +Tpm + Tmtp = 44.13 ms 83.79 ms

[13]

2Tpm = 1.92 ms

2Tbm +3Tpm = 20.04 ms

[15]

Tbm + 3Tpm + Tmtp = 28.89 ms 2Tbm +Tpm = 18.12 ms

SGAAS Tbm + 4Tpm = 12.42 ms

Tbm +4Tpm =12.42 ms

SUM 21.96 ms 47.01 ms 24.84 ms

Table 2. Comparison of communication costs Scheme Message-signature [12]

4G1 + 4 = 516 bytes

[13]

3G1 + Zq∗ = 404 bytes

[15]

3G1 + Zq∗ + 4 = 408 bytes

SGAAS 2G1 + 3Zq∗ + 4 = 320 bytes

5

Conclusion

An anonymous authentication scheme based on self-generated pseudonym for VANETs is proposed in this paper. On account of the non-interactive zeroknowledge proof mechanism, the pseudonym-based anonymous authentication protocol is implemented. The utilization of pseudonyms ensures that the real identity of the vehicle user is not compromised. Moreover, the malicious node is

An Anonymous Authentication Scheme

83

able to be traced according to the pseudonym. Through security and performance analysis, we can prove that the proposed scheme owns higher efficiency and less communication delay while meeting the privacy protection requirements in VANETs.

References 1. Qu, F., Wu, Z., Wang, F., Cho, W.: A security and privacy review of VANETs. IEEE Trans. Intell. Transp. Syst. 16(6), 2958–2996 (2015) 2. Lee, J.K., Jeong, Y.S., Park, J.H.: s-ITSF: a service based intelligent transportation system framework for smart accident management. HCIS 5(1), 1–9 (2015). https:// doi.org/10.1186/s13673-015-0054-x 3. Wei, L., Cui, J., Xu, Y., Cheng, J., Zhong, H.: Secure and lightweight conditional privacy-preserving authentication for securing traffic emergency messages in VANETs. IEEE Trans. Inf. Forensics Secur. 16, 1681–1695 (2021). https://doi. org/10.1109/TIFS.2020.3040876 4. Bayat, M., Barmshoory, M., Rahimi, M., Aref, M.R.: A secure authentication scheme for VANETs with batch verification. Wireless Netw. 21(5), 1733–1743 (2014). https://doi.org/10.1007/s11276-014-0881-0 5. Sheikh, M.S., Liang, J., Wang, W.: Security and privacy in vehicular ad hoc network and vehicle cloud computing: a survey. Wirel. Commun. Mob. Comput. 2020(3), 1–25 (2020) 6. Manivannan, D., Moni, S.S., Zeadally, S.: Secure authentication and privacy preserving techniques in Vehicular Ad hoc Network(VANETs). Vehicular Commun. 25. 100247. https://doi.org/10.1016/j.vehcom.2020.100247 7. Malhi, A.K., Batra, S., Pannu, H.S.: Security of vehicular ad hoc networks: a comprehensive survey. Comput Secur. https://doi.org/10.1016/j.cose.2019.101664 8. Azees, M., Vijayakumar, P., Deboarh, L.J.: EAAP: efficient anonymous authentication with conditional privacy-preserving scheme for vehicular ad hoc networks. IEEE Trans. Intell. Transp. Syst. 18(9), 2467–2476 (2017). https://doi.org/10. 1109/TITS.2016.2634623 9. Kumar, P., Kumari, S., Sharma, V., Li, X., Sangaiah, A.K., Islam, S.K.H.: Secure CLS and CL-AS schemes designed for VANETs. J. Supercomput. 75(6), 3076–3098 (2018). https://doi.org/10.1007/s11227-018-2312-y 10. Zhong, H., Han, S., Cui, J., et al.: Privacy preserving authentication scheme with full aggregation in VANET. Inf. Sci. 476, 211–221 (2019) 11. Wang, Y., Zhong, H., Xu, Y., et al.: Enhanced security identity-based privacypreserving authentication scheme supporting revocation for VANETs. IEEE Syst. J. 14(4), 5373–5383 (2020) 12. Wang, S., Yao, N.: LIAP: a local identity-based anonymous message authentication protocol in VANETs. Comput. Commun. 112, 154–164 (2017) 13. Liu, J., Yu, Y., Zhao, Y., Jia, J., Wang, S.: An efficient privacy preserving batch authentication scheme with deterable function for VANETs. In: Au, M.H., et al. (eds.) NSS 2018. LNCS, vol. 11058, pp. 288–303. Springer, Cham (2018). https:// doi.org/10.1007/978-3-030-02744-5 22 14. Cui, J., Tao, X., Zhang, J., et al.: HCPA-GKA: a hash function-based conditional privacy-preserving authentication and group-key agreement scheme for VANETs. Veh. Commun. 14, 15–25 (2018)

84

J. Qi and T. Gao

15. Liu, F., Wang, Q.: IBRS: an efficient identity-based batch verification scheme for VANETs based on ring signature. arXiv preprint arXiv:1909.13223 (2019) 16. Sun, M., Guo, Y., Zhang, D., et al.: Anonymous authentication and key agreement scheme combining the group key for vehicular ad hoc networks. Complexity (2021) 17. Fiat, A., Shamir, A.: How to prove yourself: practical solutions to identification and signature problems. In: Odlyzko, A.M. (ed.) CRYPTO 1986. LNCS, vol. 263, pp. 186–194. Springer, Heidelberg (1987). https://doi.org/10.1007/3-540-47721-7 12

Wavelet Transform Based PID Sequence Analysis for IDS on CAN Protocol Md Rezanur Islam1 , Insu Oh2 , Munkhdelgerekh Batzorig1 , Myoungsu Kim2 , and Kangbin Yim2(B) 1 Convergence Security, Soonchunhyang University, Asan, Korea

{arupreza,munkhdelgerekh}@sch.ac.kr

2 Department of Information Security Engineering, Soonchunhyang University, Asan, Korea

{catalyst32,brightprice,yim}@sch.ac.kr

Abstract. Due to the increasing complexity of the group of software and hardware components used in automobiles, current threats continue to hit the onboard network day after day. These additional components highlight the difficulties of developing compelling and responsive security solutions. To differentiate and defend automotive systems against deleterious exercises, a few intrusions detection systems (IDS) have been developed. Deep learning is one of the greatest options for detecting malicious packets where recurrent and convolutional neural network is vastly implemented. However, feature escalation is equally necessary for training a model. To safeguard automotive systems, we used an RNN-based LSTM algorithm as an intrusion detection system and wavelet conversion were used for feature escalation. This research emphasizes a depiction of vulnerabilities, highlights threat models, and makes it simple to recognize known threats that are displayed within the CAN.

1 Introduction In the latest years, communication and automobile technology advanced by using the blessings of the internet of things (IoT) and day-by-day communication medium and new functions integrated into cars. The IoT includes sensors, smart devices, cloud stations, and so forth. These devices are linked via numerous communication protocols physically and exchange data inside the network. With the help of new components and smart terminals, the intelligent transportation system buildup and ended up a critical application and it is connected smart vehicles electronically with street infrastructure, mobile gadgets, and the internet. Current automobiles opened many entry points for hackers with Bluetooth, mobile communique, gateways, telematics, and multiple ECUs [1]. Ethernet in cars is currently receiving quite a little attention and is utilized in the latest automobiles to over-transmit huge amounts of data with high bandwidth and very low latency and jitter [2]. This paradigm shift has extended the attack surface and safety researchers have a scope of research and present-day automobiles are no longer secure from cyberattacks [3]. Cars use a variety of electronic devices and software programs. Presently, motors are geared up with more than 70 or one hundred electronic control units (ECUs) and near approximately 2500 data to transmit internally [4] with dependable inner communique © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 85–96, 2022. https://doi.org/10.1007/978-3-031-08819-3_9

86

M. R. Islam et al.

over CAN buses and buses including C-CAN, M-CAN, and B-CAN media [5]. It’s far geared up with numerous ECUs connected in parallel. Security researchers have supplied numerous strategies to reduce cyber threats to in-vehicle networks including the entropy-based approach that measures and finds out the abnormality in CAN traffic by the usage of self-data [6]. They overlook the semantic features and are concerned about statistical functions. However semantic features should consider for more efficient and powerful intrusion detection, and network protection analysis. The time interval-based IDS method can define frequency periods of CAN messages [7]. Our preceding research represents how data label differs for exceptional sorts of assault scenario [8]. Section 2 demonstrates an entire portrayal of CAN identity and represents which way it generates data set and previous studies. Different types of attack scenarios are explained here Sect. 3. Here data generation process, experimental setup, and data conversion method are described. Section 4 mentioned RNN LSTM structure. After that in Sect. 5 our experimental result stated. Sooner or later, the future plan and conclusion are given in Sect. 6.

2 Background and Motivation 2.1 Background CAN ID is an identifier for the CAN data frame and payloads contained with this identity. Signal information is diagnosed with the aid of this identity. Due to the broadcast nature, CAN ID no longer includes which node getting this massage. But the payload which is 8 bytes hexadecimal number represents the actual commands. CAN massage is unique for each car version even for automobiles of the identical manufacturer [9]. Producers preserve those records immensely confidential but reverse engineering and data analysis can make them partially accessible (Fig. 1).

Fig. 1. Data frame of CAN protocol

The controller area network broadcast messages and all connected nodes receive the messages. CAN bus frame consists of 4 sorts [10]: data frame, error frame, overload frame, and remote frame. (1) Data frame: This is the only frame used for payload transfer. (2) Remote frame: This frame is solely used to request payload transfer. When an ECU gets a remote frame, it replies instantly. (3) Error frame: This frame defines and examines if there is an error. (4) Overload frame: This frame is used to postpone the commencement of the next message if there is an overload. Now we describe the payload of CAN massage which is mainly a signal produced by ECUs. For communicating with each other all eight bytes are not used by ECUs. Those have few segmentations like Constants: Constant values remain unchanged over time.

Wavelet Transform Based PID Sequence Analysis for IDS

87

Multi-Values: A few bytes responsible for the payload of the command are called multivalues [11], reported 2–3 changing values within these types of signals. An instance of a 2-value field will be responsible for the selected door for open and closed. Counters: counters are indicators that behave as cyclic counters inside a selected range. These indicators could function as additional syntax checks or be intended to order longer sign records at the destination ECU(s). Whilst this value is an internal positive range, it turns agitated, just like the speed value. Now we describe the payload of CAN massage which is mainly a signal produced by ECUs. Checkcodes: Except for the CRC-15 field at the end of each CAN body, the payload also can contain extra checkcodes, typically as the last signal within the payload. 2.2 Previous Work and Assault Scenario In this phase, we talk about the essence of the related work concerning network anomaly detection of in-car systems. The number one to demonstrate the assault injection through wi-fi communication in-car systems. Koscher et al. become an investigation the security of contemporary vehicles [12]. By way of the usage of CARSHARK device, they collect all kinds of records and due to the broadcasting manner on CAN, attackers can easily have an effect on CAN communication. Open Car Testbed and Network Experiments (OCTANE) is a kind of device-generated data similar to the car packet. In this paper [13], the author implements this device and investigated various styles of assaults on related automobiles which provided an overview of the artificial neural network (ANN) - based IDS to protect against cyber attacks on modern-day vehicles systems. Javed AR et al. proposed the viability of evaluation approaches to detect intruders within the vehicle network [14]. They converted records points into a vector sequence to be fed into the CNN layers. DBN-based IDS for the CAN BUS IDS was provided with the aid of Kang et al. [15]. It provides an unsupervised deep belief network (DBN). LSTM model applied by Taylor et al. [16] Their concept is based on the prediction of the following statistics of the CAN bus network whilst acknowledging that the statistics originate from the senders. Kleberger et al. mentioned protection features architecture and afterward mentioned the problems and solutions [17]. The wireless attack was initiated with the aid of Woo et al. [18]. They stated the connected vehicle environment, numerous forms of assault models, and protection necessities. Luo J-N et al. [32] proposed an authentication mechanism for in-vehicle networks where they replaced CRC fields with MAC. Khan et al. investigated SDN-based false data injection into the brake-associated ECUs. They developed a fake data assault dataset and used LSTM to hit upon the attack, and they completed a detection rate of 87% [19]. Cloud-primarily based attack on the robotic vehicle made by Loukas et al. [20]. They used multiple machine learning classifiers. Researchers in [21] Jaynes et al. used a proprietary machine-learning set of policies to classify CAN bus messages. Their consequences display that the k-nearest-neighbor (k-NN) set of rules is more entire with 86.00% accuracy. Lee et al. [10] developed an IDS for reading request/response messages in the CAN bus, primarily based on offset ratio and time interval reviews. In this section, we are discussing our implemented assaults fuzzing, DoS and replay.

88

M. R. Islam et al.

Fuzzing Attack. In this elegance of assaults, the attacker makes use of any spoofing identity that includes subjective data to compose the message. In this situation, all hubs will receive an unusually useful spoofed message which is randomly created packet IDs. In this case, all nodes acquire anomalous anomalies [22]. To release a fuzzing assault, an attacker ought to first watch car internal massage and pick a goal. The fuzzing assault is basically produced at a slower rate than the DoS assault [23]. Be that as it could, it is plausible to carry out a fuzzing assault at a higher rate. Dos Attack. High priority packets are injected by using an attacker in a short time interval on the bus and keep the bus busy. Typically, the attacker uses one or more excessive-priority IDs to generate DoS assault. Consequently, specific low-precedence nodes cannot get proper access to the network. All hubs share a single bus, increasing higher volume of data at the bus can create a delay or deny entries of valid packets example mentioned in [24]. The DoS attack can purpose a vehicle not to answer the driving force’s commands on time. Replay Attack Class. At first, attackers carefully observed the running massage and processed all the massage. Each payload contains important CAN message management. Therefore, the vehicle may experience manageable damage or surprising behavior. Replay attacks are one of the most important hard attacks to discover [25]. An attacker intercepts network traffic and excludes data originating from one or more randomly selected target node IDs. The attacker saves these data along with the actual packet access time. It is used later to accurately mimic or replay by injecting these packets into the network.

3 Data Description 3.1 Data Collection and Setup Most of the researcher’s interfering device is connected to an OBDII connector after that the analyzer can find available PID (parameter ID). But our process is different. We collect data from the internal gateway where all integrated ECUs are connected. In the gateway, we tap interfering devices specifically on CAN High and CAN Low and collect row data. The main difference is between the OBUII port, and our method that, in the OBUII port all ECUs are not connected therefore all types of data cannot be captured on the other hand in the internal gateway all ECUs data can be captured. Raw CAN traffic includes both diagnostic response messages and regular CAN traffic messages. Here, we analyze the raw CAN traffic and extract candidates that have the same value as the diagnostic response. The CAN specifications vary by vehicle model, but the PID is the same as defined in the J1979 standard [9]. Vehicle status data can be collected using standard PIDs, regardless of vehicle model. As an interfacing device, we use the PEAK CAN system. For our investigation, we use python Jupyter and Keres with TensorFlow as the backend. We conduct our experiment with Intel(R) Core (TM) i9-10900K CPU @ 3.70 GHz 3.70 GHz and NVIDIA GeForce RTX 2080 super.

Wavelet Transform Based PID Sequence Analysis for IDS

89

3.2 Wavelet Transform and Feature Extraction Wavelets are a popular tool for computational harmonic analysis. A notable feature is the functionality to carry out a multiresolution analysis [26]. Wavelets are absolutely suited for defining multiresolution capabilities. Those sparse representation belongings are key to the first-rate general performance of wavelets in applications including data compression and denoising. PyWavelets is a Python package that specifies some of the n-dimensional discrete wavelet transforms similarly to the 1D continuous wavelet transforms. All multidimensional modifications are implemented in Python via software that is separable from 1D transformations. PyWavelets had been designed for use with the aid of scientists working on a ramification of packages inclusive of time series evaluation, signal processing, image processing, and medical imaging [27]. The peaks in the frequency spectrum propose the maximum occurring frequencies within the signal. The bigger and sharper a height is, the greater commonplace a frequency is in a signal. The location (frequency-value) and height (amplitude) of the peaks within the frequency spectrum may be used as entering for classifiers. This easy technique works pretty properly for plenty of class issues. Wavelets had been able to classify the human activity recognition dataset with a 91% accuracy [28]. Due to the fact, that most of the indicators we see in real life are non-desk bound in nature. Approximately ECG indicators, the stock marketplace, device or sensor data, and so on, and many others, in actual-lifestyles problems, begin to get complex whilst all are coping with dynamic systems. A much higher technique for studying dynamic indicators is to use the Wavelet transform. The wavelet redesign has an excessive choice in each frequency and the time domain. It does no longer only inform us which frequencies are observed in a signal but additionally, at which period those frequencies have passed off. PyWavelets begin at the beginning of the character and slowly flow into wavelets closer to the end of the signal.

Fig. 2. Daubechies wavelet multiplier.

This process is also known as convolution. Wavelet transform is that there are numerous one-of-a-kind families (types) of wavelets. We would select a selected wavelet’s own family which suits best the capabilities we’re seeking out in our signal. Each kind of wavelet has a different shape, smoothness, and compactness and is beneficial for a particular data. A wavelet can be complex or actual. If it is complex, it’s also divided into a real element representing the amplitude and an imaginary detail representing the phase.

90

M. R. Islam et al.

Here, we use one family of wavelets known as ‘Daubechies’ shown in Fig. 2. It is an orthogonal wavelet that needs time-frequency localization. Wavelet coefficient shows approximation coefficients vector and detailed coefficients vector. Here is used CAN ID and 8 bytes of hexadecimal data where every single byte is a PID. In CAN protocol single PID or combine PID carry out the functional massage and ECUs get command and apply all of the operations through this PID. As a result, PID changing pattern played an important role in detecting CAN vulnerability. A total of 1200000 data was captured. This data set was split by 10000 and shuffled for making the data set more challenging. The deep-learning algorithm cannot deal with characters and CAN ID and PID is character categories. For that reason, by using a label encoder, we give a numerical name for every character and convert the data set into the spectrum frequency domain and this wavelet conversion represents which time which frequency exists. In our data set, there is used label 2 which returns the double-level discrete wavelet transform (DWT) of the vector of x and as a result, it gives the coefficient cA1, cD2, cD1. cA represents the approximation coefficients vector and cD detail coefficients vector. cA1 and cD2 give three features and cD1 gives five features as a result total features of this data set are nine.

4 Deep Learning Model and Architecture In RNN, LSTM is the most popular neural network that included special units referred to as memory blocks within the recurrent hidden layer. Memory blocks are self-connected cells, and they save the initial state of the network after it modifies the flow of the input in large multiplication units named gates. Each memory blocks have an entry port and exit port. The input gate controls the flow that enters the activation into the memory cell. The output gate controls the output flow of cell activations into the rest of the network. Afterward, in forget gate, all the memory block is connected [29]. The internal state of the cell is the forgetting gate scale, and it adds it as input to the cell by the cell’s self-repeating connection. Table 1. Deep learning algorithm parameters Parameter

Values

Activation function

tanh

Activation function output

Softmax

Output layer cells

4

Optimizer

RMSprop

Learning rate

0.0001

Batch size

128

Loss function

CategoricalCrossentropy

Dropout

0.2

Encoder

Label encoder

Wavelet Transform Based PID Sequence Analysis for IDS

91

This adaptively forgets or resets the cell’s memory. In addition, for learning the right timing of the outputs, the modern LSTM structure consists of peephole connections from its inner cells to the gates inside the identical cell [17]. Deep LSTM RNNs are built by stacking multiple LSTM layers. Note that LSTM RNNs are already deep neural network architectures withinside the experience that they will be taken into consideration as a feed-in advance neural network unrolled in time wherein every layer shares the same version parameters. Like a deep neural network models inputs undergo a couple of non-linear. However, features at a particular point in time can most effectively be processed by using a single non-linear layer earlier than contributing to the output at that factor in time. Because of this, depth has a special that means in deep LSTM RNN. Inputs to the network at a particular time step do not propagate perfectly on the time LSTM layers and same as at additionally through multiple LSTM layers. There are distinctive thoughts about deep layers in RNNs permit the network to learn at different time scales over the input [18]. Deep LSTM RNNs provide some other advantages over standard LSTM RNNs: They are able to make higher use of parameters by using dispensing them over the gap using multiple layers. As an example, the model doesn’t increase the memory size of the model, it remains approximately the same number of parameters. Our LSTM model consists of four hidden layers where used activation function as tanh and output activation function as softmax included with four neurons. Every hidden layer represents 120 neurons with dropout and batch normalization. Optimizer RMSprop with a learning rate of 0.0001 gave a better result. Table 1 mentioned the detailed brief.

5 Experiment Result and Performance Evaluation There are so many patterns already found after data analysis. In specific CAN ID, the data injection time gap is almost the same. For example, in fizzing and DoS attack a set of CAN ID used for injecting high-frequency data. Another finding is, that here is used BMW car data set, where 56 CAN ID operates all of the internal commands in the car. But when the fuzzing attack appeared 1496 CAN ID was generated and in the DoS attack, the ID number was totally inverse from fuzzing. The number is 41, in a replay attack number of CAN ID remains the same as anticipated. After wavelet conversion data is arranged in a manner it can specify the variation of the data, the x-axis represents time, and the y-axis is frequency. The pattern between normal and attack mode can be differentiated easily. Though normal and replay attack have small difference and their frequency amplitude are identical -200 to 400 but in replay attack amplitude spikes define accurately this is not a normal data set, whereas normal data follow a periodic manner shown in Fig. 3(a). Fuzzing attack frequency range between −1000 to 1500 which is higher than normal mode and in the replay there is missing the smoothness of frequency amplitude represented in Fig. 3(b). DoS attack frequency amplitude is lower than normal Fig. 3(c), the range is −100 to 200. In this experiment, the LSTM model is used, and LSTM needs 3dimensional data as input, and we used here a time-series pattern because CAN massage is a time series data. At a time, it will take ten data sets and train the model. We reshape the data into 1200000, 10, 11, and feed the model. Our overall accuracy is 99.98% and

92

M. R. Islam et al.

Fig. 3. The portrayal of normal and attack scenarios (a) Normal, (b) Fuzzing, (c) DoS, (d) Replay after wavelet conversion.

our ROC score is 0.9985341. Table 2 and confusion matrix Fig. 4(a) gives an overview of the result evaluation. Table 2. Performance evaluations for implemented model. Data type

Precision

Recall

F1-score

Total test data

Normal

0.99

1.00

1.00

11487

Fuzzing

1.00

1.00

1.00

36808

DoS

1.00

1.00

1.00

9231

Replay

1.00

0.97

0.98

2474

Accuracy

NA

NA

1.00

60000

Macro avg

1.00

0.99

1.00

60000

Weighted avg

1.00

1.00

1.00

60000

Overall score

Wavelet Transform Based PID Sequence Analysis for IDS

93

The model can accurately identify Fuzzing and DoS attack, the performance rate almost is 100% because the Fuzzing and DoS attack has their own characteristics, and wavelet conversion identifies characteristics perfectly. In a replay attack, our model identifies a few amounts of normal data as a replay attack, but the overall performance is 97%. The reason is assaulter initiates a replay attack by actual data of the victim’s car. Figure 4(b) represents the ROC score for the multiclass classification. ROC curve is a performance measurement mechanism for classification problems at various threshold settings. ROC is a probability curve and AUC represents the degree or measure of separability. It tells how much the model is capable of distinguishing between classes. In our ROC curve normal, fuzzing and DoS attack are classified perfectly because everyone has their own pattern but replay attack AUC is 98% because of the similarity with normal data.

Fig. 4. Evaluation score (a) Confusion matrix, (b) ROC score.

94

M. R. Islam et al.

Fig. 4. continued

6 Conclusion Even though the automobile trying-out era has made exquisite progress, there is still a lack of relevant vehicle safety assessment tools inside the market. Existing testing tools additionally have problems. For that reason, a research scope opened for the researcher. We are employed here in an LSTM model. Other researchers have employed LSTM before, but their input data is different. Here we tried to find out data changing patterns in the frequency domain for different types of attacks and used core data from a registered car. In this investigation, we used statistical data to detect malicious data for BMW model car and it is perfectly identifying the malicious. Assaulters use different types of attack, but they cannot maintain normal mode frequency patterns and wavelets specify all of the details. Which make our IDS specialty that can differentiate normal and abnormal situation for every single portion. Our future goal is by analyzing all features of CAN massage to make a universal IDS that can be employed for all types of cars. Acknowledgments. This work was supported by the National Research Foundation of Korean (NRF) grant funded by the Korean government (MSIT) (No. 2021R1A4A2001810) and the Institute for Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korean government (MSIT) (No. 2019-0-01343, Regional strategic industry convergence security core talent training business).

Wavelet Transform Based PID Sequence Analysis for IDS

95

References 1. Kelarestaghi, K.B., Foruhandeh, M., Heaslip, K., Gerdes, R.: Intelligent transportation system security: impact-oriented risk assessment of in-vehicle networks. IEEE Intell. Transp. Syst. Mag. 13(2), 91–104 (2021). https://doi.org/10.1109/MITS.2018.2889714 2. Carnevale, B., Fanucci, L., Bisase, S., Hunjan, H.: MACsec-based security for automotive ethernet backbones. J. Circuits Syst. Comput. 27(05), 1850082 (2018). https://doi.org/10. 1142/S0218126618500822 3. Checkoway, S., et al.: Comprehensive experimental analyses of automotive attack surfaces. In: Proceedings of the 20th USENIX Security Symposium, pp. 77–92 (2011) 4. Hassija, V., Chamola, V., Saxena, V., Jain, D., Goyal, P., Sikdar, B.: A survey on IoT security: application areas, security threats, and solution architectures. IEEE Access 82721–82743 (2019). https://doi.org/10.1109/ACCESS.2019.2924045 5. An, Y., Park, J., Oh, I., Kim, M., Yim, K.: Design and implementation of a novel testbed for automotive security analysis. In: Barolli, L., Poniszewska-Maranda, A., Park, H. (eds.) IMIS 2020. AISC, vol. 1195, pp. 234–243. Springer, Cham (2021). https://doi.org/10.1007/978-3030-50399-4_23 6. Muter, M., Asaj, N.: Entropy-based anomaly detection for in-vehicle networks. In: 2011 IEEE Intelligent Vehicles Symposium (IV), pp. 1110–1115, June 2011. https://doi.org/10. 1109/IVS.2011.5940552 7. Song, H.M., Kim, H.R., Kim, H.K.: Intrusion detection system based on the analysis of time intervals of CAN messages for in-vehicle network (2016). https://doi.org/10.1109/ICOIN. 2016.7427089 8. Islam, M.R., Oh, I., Batzorig, M., Kim, S., Yim, K.: A concept of IDS for CAN protocol based on statics theory. In: Barolli, L. (ed.) BWCCA 2021. LNNS, vol. 346, pp. 294–302. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-90072-4_32 9. Song, H.M., Kim, H.K.: Discovering CAN specification using on-board diagnostics. IEEE Des. Test 38(3), 93–103 (2021). https://doi.org/10.1109/MDAT.2020.3011036 10. Lee, H., Jeong, S.H., Kim, H.K.: OTIDS: a novel intrusion detection system for in-vehicle network by using remote frame. In: 2017 15th Annual Conference on Privacy, Security and Trust (PST), pp. 57–5709. IEEE (2017). https://doi.org/10.1109/PST.2017.00017 11. Markovitz, M., Wool, A.: Field classification, modeling and anomaly detection in unknown CAN bus networks. Veh. Commun. 9, 43–52 (2017). https://doi.org/10.1016/j.vehcom.2017. 02.005 12. Koscher, K., et al.: Experimental security analysis of a modern automobile (2010). https:// doi.org/10.1109/SP.2010.34 13. Haas, R.E., Moller, D.P.F., Bansal, P., Ghosh, R., Bhat, S.S.: Intrusion detection in connected cars. In: 2017 IEEE International Conference on Electro Information Technology (EIT), pp. 516–519, May 2017. https://doi.org/10.1109/EIT.2017.8053416 14. Javed, A.R., Rehman, S.U., Khan, M.U., Alazab, M., Reddy, T.: CANintelliIDS: detecting in-vehicle intrusion attacks on a controller area network using CNN and attention-based GRU. IEEE Trans. Netw. Sci. Eng. 8(2), 1456–1466 (2021). https://doi.org/10.1109/TNSE.2021. 3059881 15. Kang, M.-J., Kang, J.-W.: Intrusion detection system using deep neural network for in-vehicle network security. PLoS ONE 11(6), e0155781 (2016). https://doi.org/10.1371/journal.pone. 0155781 16. Taylor, A., Leblanc, S., Japkowicz, N.: Anomaly detection in automobile control network data with long short-term memory networks. In: 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA), pp. 130–139, October 2016. https://doi.org/ 10.1109/DSAA.2016.20

96

M. R. Islam et al.

17. Kleberger, P., Olovsson, T., Jonsson, E.: Security aspects of the in-vehicle network in the connected car (2011). https://doi.org/10.1109/IVS.2011.5940525 18. Woo, S., Jo, H.J., Lee, D.H.: A practical wireless attack on the connected car and security protocol for in-vehicle CAN. IEEE Trans. Intell. Transp. Syst. 1–14 (2014). https://doi.org/ 10.1109/TITS.2014.2351612 19. Khan, Z., Chowdhury, M., Islam, M., Huang, C.-Y., Rahman, M.: Long short-term memory neural networks for false information attack detection in software-defined in-vehicle network, June 2019. http://arxiv.org/abs/1906.10203 20. Loukas, G., Vuong, T., Heartfield, R., Sakellari, G., Yoon, Y., Gan, D.: Cloud-based cyberphysical intrusion detection for vehicles using deep learning. IEEE Access 6, 3491–3508 (2018). https://doi.org/10.1109/ACCESS.2017.2782159 21. Jaynes, M., Dantu, R., Varriale, R., Evans, N.: Automating ECU identification for vehicle security (2017). https://doi.org/10.1109/ICMLA.2016.53 22. Lee, H., Choi, K., Chung, K., Kim, J., Yim, K.: Fuzzing CAN packets into automobiles. In: 2015 IEEE 29th International Conference on Advanced Information Networking and Applications, pp. 817–821, March 2015. https://doi.org/10.1109/AINA.2015.274 23. Nowdehi, N., Aoudi, W., Almgren, M., Olovsson, T.: CASAD: can-aware stealthy-attack detection for in-vehicle networks, September 2019. http://arxiv.org/abs/1909.08407 24. Murvay, P.-S., Groza, B.: DoS attacks on controller area networks by fault injections from the software layer. In: Proceedings of the 12th International Conference on Availability, Reliability and Security, pp. 1–10, August 2017. https://doi.org/10.1145/3098954.3103174 25. Hoppe, T., Kiltz, S., Lang, A., Dittmann, J.: Exemplary automotive attack scenarios: trojan horses for electronic throttle control system (ETC) and replay attacks on the power window system, VDI Berichte, pp. 165–183 (2007) 26. Mallat, S.: A Wavelet Tour of Signal Processing. Elsevier, Amsterdam (2009) 27. Lee, G., Gommers, R., Waselewski, F., Wohlfahrt, K., O’Leary, A.: PyWavelets: a python package for wavelet analysis. J. Open Source Softw. 4(36), 1237 (2019). https://doi.org/10. 21105/joss.01237 28. Taspinar, A.: A guide for using the wavelet transform in machine learning (2018). https://ata spinar.com/ 29. Gers, F.A., Schraudolph, N.N., Schmidhuber, J.: CrossRef List. Deleted DOIs, vol. 1 (2000). https://doi.org/10.1162/153244303768966139 30. Gers, F.A., Schmidhuber, J., Cummins, F.: Learning to forget: continual prediction with LSTM. Neural Comput. 3, 115–143 (2000). https://doi.org/10.1162/089976600300015015 31. Hermans, M., Schrauwen, B.: Training and analyzing deep recurrent neural networks. In: Advances in Neural Information Processing Systems, 2013. Appendix: Checklist of Items to be Sent to Conference Proceedings Editors (see instructions at conference webpage), pp. 190– 198 (2013) 32. Luo, J.-N., Wu, C.-M., Yang, M.-H.: A CAN-bus lightweight authentication scheme. Sensors 21(21), 7069 (2021). https://doi.org/10.3390/s21217069

Efficient CAN Dataset Collection Method for Accurate Security Threat Analysis on Vehicle Internal Network Yeji Koh, Seoyeon Kim, Yoonji Kim, Insu Oh, and Kangbin Yim(B) Department of Information Security Engineering, Soonchunhyang University, Asan, Korea {julysnowflake,seoyeon56,rladbswl7,catalyst32,yim}@sch.ac.kr

Abstract. As the rapid growth of vehicle functions and safety technologies, the types of ECU (Electronic Control Unit) mounted inside the vehicle has increased, and accordingly, for the safety and security of CAN (Controller Area Network), a protocol to ensure the secure communication between ECUs is required. Due to the increasing need for research and analysis related to CAN protocol, various and accurate vehicle CAN data sets are required than ever before. In this paper, we propose more efficient and accurate data collection method for better analyze of CAN message, and based collected data, we have analyzed and made results using the amount of data, time intervals, and various characteristics from each method. The collected in-vehicle network datasets through this research are expected to be utilized for diverse security threat analysis.

1 Introduction In modern times, a vehicle’s functions and safety technologies are growing rapidly. For the safety and convenience of drivers, technologies like autonomous driving car that can self-driving without human driver, and V2X technology that used in communication among vehicles and people have been greatly developed. As result, various connections are required to vehicles, and this leads to increase of cybersecurity threats to vehicles and internal networks. The CAN (Controller Area Network) protocol is a de-facto network protocol that has the most important role in the communication process between vehicle ECUs. The low cost, relativity high reliability, and fault tolerance properties of CAN is the main reason to be used in most in-vehicle communication, and they are also actively used in the process of realizing various functions. To complete those performance, the CAN protocol uses broadcast message as communication, and this makes the CAN protocol vulnerable to network attacks. Recently, a considerable attention has been paid to reactive systems, such as Intrusion Detection Systems (IDSs), as solution to those cyber-attacks. At the same time, to make more accurate analysis, it demands more accurate dataset with high relatability. Thus, collecting more diverse and accurate vehicle CAN data has become more crucial than ever before. In this paper, we focused on devising a more efficient and accurate vehicle CAN data collection method. Furthermore, we collect data using collection tools that mentioned other various papers and compared with our method. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 97–107, 2022. https://doi.org/10.1007/978-3-031-08819-3_10

98

Y. Koh et al.

The remaining of this paper is organized as follows: Sect. 2 reviews research that related to CAN data collection method. Section 3 gives an overview of general CAN data collection method. After shows difference of each collection tools from Company A, Company B and OBDII which is a tool CAN data collection, then analyzes the collected data using the tools. Section 4 explains the EDA (CAN gateway ECU Direct Object), the new method for CAN data collection which is directly connects to the gateway. Section 5 compares the data analysis results of each method, and then evaluates efficiency for data collection. Finally, Section 6 presents analysis results and conclusions.

2 Related Works Andrew Tomlinson et al. present a CAN broadcasting analysis and a test of statistical methods to detect time changes in CAN traffic. [1] They recorded CAN messages using the Kvaser Leaf Light1 reader, and using OBDII port, they have analyzed the initial CAN of the vehicle with the captured CAN data. It introduces an intrusion detection system using hybrid techniques that can detect attack signatures and abnormal events at the same time. [2] Hyun Min Song et al. collected data from an real vehicle’s OBDII port by connecting this port to a laptop. As a result of the experiment, their proposed IDS succeeded in detecting a message injection attack with 100% accuracy. In addition, Fang Zhou et al. proposes a method of developing a simulation and test system for a body CAN network. [3] They also used Vector CANoe tool for simulation, and successfully analyzed data frame by monitoring communication situation. Adrian Taylor et al. evaluate the efficiency of frequency-based anomaly detection against packet injection attacks. [4] They collected and tested CAN bus data in Ford Explorer car. As a result, they confirmed that most attacks were detectable but not additional packets inserted. In addition, Habeb Olufowobi et al. develop SAIDuCAnt, a standard-based intrusion detection system that inputs real-time models [5]. They evaluate SAIDuCAnt using log datasets and open-source CAN data sets extracted from OBDII ports of two vehicles. Attacks could be detected with high accuracy and reduced detection delays compared to other detection techniques. Hongmao Qin et al. propose two data types of anomaly detection algorithms to detect abnormal behavior of CAN buses using the LSTM algorithm. [6] For the experiment, they connected to the OBDII port to capture CAN data packets and monitored and analyzed CAN message flows using Vehicle Spy. The proposed algorithm has been proven to perform through testing and evaluation with an accuracy of 90% or more. A deep learning-based approach is proposed to detect cyber-attacks on vehicles. [7] Abdollah Kavousi-Fard et al. recorded datasets using the OBDII port to evaluate the performance of the proposed model, and experiments show that anomaly detection models can block multiple cyber-attacks on CAN buses (Table 1).

Efficient CAN Dataset Collection Method for Accurate Security

99

Table 1. CAN data collection method and scope of utilization Ref

CAN collection method (Tool)

Data utilization field

[1]

OBD (Kvaser Leaf Light1)

- Detecting the time change of CAN traffic - Analysis of the initial CAN data of the vehicle

[2]

OBD

- Manufacture of vehicle CAN network IDS - Message injection attack detected successfully

[3]

Vector CANoe

- Monitoring and testing the CAN network communication status

[4]

OBD

- Evaluation of the efficiency of frequency-based anomaly detection

[5]

OBD

- Evaluate the developed SAIDuCANT

[6]

OBD (Vehicle Spy)

- Monitor and analyze CAN message flow

[7]

OBD

- To evaluate the performance of the proposed model

3 Common CAN Data Collecting Methods 3.1 CAN (Controller Area Network) Since the 1980s, the electronic control unit (ECU) has been introduced into vehicles due to the increase in electronic devices. It is connected to the network to share the necessary data between ECUs that control all devices such as engines and transmissions, and IVN (In-Vehicle Network), a network for interior vehicles, IVN has been developed to identify specificities and problems in the vehicle [8]. Initially, UART (Universal Asynchronous Receiver/Transmitter), an asynchronous serial communication method, was used, which required more connections every time a module was added because each module had one-on-one communication, leading to weight and cost increase. To solve this problem, a representative bus system, the Controller Area Network (CAN) protocol, was developed in 1985. CAN was designed for controller devices to communicate without a host computer in a vehicle and was established by ISO in 1993 as one of the most widely used standard communication specifications in vehicle internal communication [8]. Because CAN communication uses two signals CAN_High and CAN_Low, the number of lines is smaller than the number of additional modules, which is resistant to electrical noise, and can safely protect messages. In addition, one CAN interface with a parallel structure allows control of multiple ECU modules, providing efficient and PLUG&PLAY functions, making it easy to add and remove CAN controllers to buses, making it easy to expand ECU. Furthermore, data that transmitted in vehicle can be easily accessed on CAN buses with a multi-master network (multi-network) method,

100

Y. Koh et al.

and the Carrier Sense Multi-Access/Collision Detection (CSMS/CD) method and the Arbitration on Message Priority (AMP) method are also used for data collection [8]. 3.2 Common CAN Data Collection Method and Collection Tool This chapter describes OBDII, a CAN data packet collection method that has been used in the data collection process of a typical vehicle, and data collection tools of Device A and Device B used in the CAN data collection process. OBDII stands for On Board Diagnostics and is a computer-based early diagnosis system built into a vehicle. It was used to check and control the electrical/electronic operation state of a vehicle and was initially used to improve the efficiency of vehicle maintenance, but it currently serves as an interface that shows various vehicle information to a driver in addition to the present purpose. The OBDII port has a built-in CAN port related to diagnosis, and through that CAN port, many analysts access and analyze the CAN bus data of the vehicle. However, there is only CAN for diagnosis in the OBDII port, and different CANs are located depending on the vehicle type, so vehicle data is limited. In the case of the vehicle under analysis, Chassis and Multimedia-CAN are located, and in the vehicles manufactured after 2019, there is a Diagnostics-CAN for diagnosis only (Fig. 1).

Fig. 1. OBDII port

The OBDII port is located at the bottom of the driver’s seat or under the steering wheel of the vehicle, and CAN data can be collected by installing an OBD connector in the OBDII port and connecting it with a CAN data collection tool. The OBDII port may collect diagnosis-related CAN data, and the collected CAN classification name may vary depending on the vehicle. If OBDII connectors are connected to Device A and Device B, respectively, and set at 500 Kbit/s speed of 24 Hz, vehicle data can be confirmed to enter normally (Fig. 2). Device A, a CAN-USB connection tool, is a tool that connects the vehicle’s CAN bus and all computers with USB connectors. This tool allows CAN network packet messages to be injected or CAN Bus traffic to be recorded without additional Internet connection. It supports CAN transmission speed and CAN 2.0 specification of up to 1 Mbit/s. However, additional CAN analysis software provided by the company is needed to check CAN message packets.

Efficient CAN Dataset Collection Method for Accurate Security

101

Fig. 2. The process of collecting CAN data using OBDII

Device B, a CAN-USB connection tool, is a tool that connects the vehicle’s CAN bus and computer, just like the Device A. Unlike Device A, this tool requires additional Internet connections and certificates, but Ethernet, not only CAN networks, can also be analyzed. This tool can record CAN Bus traffic and test through virtual ECU configuration. 3.3 Connecting Device A to OBDII CAN data collection was conducted by connecting OBDII, a diagnostic CAN data collection tool located inside the vehicle, and Device A. Driving data were collected within the same time for a certain period, and functional aspects such as obstacles and direction indicators were also carried out to be as consistent as possible as before. When OBDII collects packets using Device A, it was confirmed that the total number of data collected in 358 s was 749,126, and the time difference between the collected data averaged 0.00047 s, up to 0.0046 s and at least 0.0001 s intervals (Fig. 3).

Fig. 3. The amount of OBDII CAN data per hour collected using device A

102

Y. Koh et al.

3.4 Connecting Device B to OBDII A total of 749,703 data could be collected by connecting to Device B using the OBDII connector at the OBDII port and driving a certain section. The difference between the amount of data obtained by Device A was about 500 or more narrow values, which were more than the amount of data from Device B, but were not significantly affected. In addition, it was confirmed that the interval between the data was about 0.0002 s to up to 0.0031 s or more, and there was no significant difference in performance compared to Device A. It may be seen that the overall frame structure of the Device B is the same as the Device A, but the frame information of Device B is more specific (Fig. 4).

Fig. 4. The amount of OBDII CAN data collected per hour using device B

4 New Methods for Collection CAN Data 4.1 EDA: CAN Gateway ECU Direct Approach

Fig. 5. The process of collecting CAN data using EDA

CAN gateway ECU means an ECU accessible to a CAN network. The method used in this paper is to access the CAN Gateway ECU and collect data through ECU’s line tapping using tools, which is defined as the CAN Gateway ECU Direct Approach (EDA).

Efficient CAN Dataset Collection Method for Accurate Security

103

As such, there is an ECU that can access the CAN network inside the vehicle, typically an integrated central control unit (ICU). When data is collected using the EDA method, CAN data collection of more diverse functions is possible because the entire CAN bus of the vehicle is accessible in addition to the diagnostic CAN data. However, since the location of major ECUs inside the vehicle varies from vehicle to vehicle of each manufacturer, it is highly likely that they will experience difficulties in accessing data collection, so we disassemble the vehicle, and able to find the gateway by using manual script of the vehicle. Figure 6 is the result of applying the CAN data collection process to the actual car described in Fig. 5. In the case of the vehicle, it was easier to access the AVN than the ICU, and it was possible to collect CAN data as much as the ICU, so the ECU was designated as a gateway ECU to access the CAN network. B-CAN, C-CAN, and M-CAN were located as shown in the left picture of Fig. 6, and it was verified that they could be accessed, as shown in the right picture.

Fig. 6. How to secure a dataset through CAN gateway

4.2 Connecting Device A to Gateway CAN data collection of a vehicle is performed while driving in the same manner as the collected data using OBDII. When packets are collected using Device A by tapping specific CAN lines from major ECUs, the total number of data collected during that time is 747,489. The time difference between the collected data was between 0.0002 and 0.0003 s on average, and in the maximum case, a difference of 0.003 s or more occurred (Fig. 7).

104

Y. Koh et al.

Fig. 7. The amount of EDA CAN data collected per hour using device A

4.3 Connecting Device B to Gateway When packets are collected using the Device B, the number of data collected during the same time is 749,463. A small amount of data was additionally collected according to the difference in detailed seconds, but as a result, approximate data were collected. The time difference of the data collected through Device B averaged between 0.0002 and 0.0003 s, and in the maximum case, there was a difference of more than 0.0015 s (Fig. 8).

Fig. 8. The amount of EDA CAN data collected per hour using device B

5 Evaluation For efficient CAN data collection, we analyzed the differences in data result values according to CAN data collection tools and methods. The target vehicle is a Company A’s 2016 vehicle, and a study was conducted on the C-CAN of the vehicle. Data used

Efficient CAN Dataset Collection Method for Accurate Security

105

in the analysis was collected for 358 s, and vehicle driving data were collected using the same route. In the case of functional operation of the vehicle while driving, only the same function as possible is operated, and only differences in data collection methods and tools exist.

Fig. 9. Comparison graph of data collection by collection method (Company A)

Overall, the difference in the process of data collection according to the CAN data collection tool was narrow, and no significant difference in the number and time difference was found between data on CAN lines directly tapped to OBDII and ECU. Figure 9 shows that a result of measuring the amount of data coming in at 10-s intervals, no specificity was found in Device A and Device B, and both collection tools were narrow, but more data was collected using EDA (Fig. 10).

Fig. 10. Comparison graph of data collection by collection method (Company B)

For a clear comparison, a data collection analysis between the EDA method and the OBDII port was conducted on a Company B’s 2014 vehicle, which is only a diagnostic CAN located on the OBDII port, as a result, it showed a clear difference from Company A’s 2016 vehicle in terms of data collection. The amount of collecting data was measured at intervals of 5 s, and it could be confirmed that more than 4000 data were collected for the EDA method, but only about 50 data were collected for OBDII.

106

Y. Koh et al.

Vehicles differ in the type and amount of data collected in OBDII depending on the year or vehicle type. The vehicle we used for our study was a Company A’s 2016, and Chassis-CAN and Multimedia-CAN were identified when looking at the circuit diagram of the OBDII port. Therefore, there is a small difference between the amount of C-CAN data extracted by the EDA method and the amount of data collected by the OBDII port. However, in the case of diagnostic data collected using the OBDII port, such as analysis using Company B’s 2014 vehicle, only data packets related to diagnosis can be collected, but when collected using the tapping method, CAN classified for each function, such as Body-CAN and Chassis-CAN, can be collected respectively. If each CAN data packet is collected in that way, more diverse CAN data can be obtained than those collected at the OBDII port, and can be used for in-depth analysis of a specific CAN or for conducting any attack test through that CAN.

6 Conclusion In most studies that related to internal networks and vulnerabilities of vehicles, CAN data is collected using OBD-II ports. When CAN data is obtained using the OBDII port, only limited data can be collected, making it difficult to identify data that generated during the communication between ECUs of the vehicle. Therefore, in this paper, we proposed EDA (CAN gateway ECU Direct Object), a new method for data collection with no limitation on collected data. In EDA, we directly connect to the gateway ECU and collect data directly from the CAN line for each function. In the case of this method, detailed analysis, and attack of CAN data responsible for a specific function such as a vehicle body and multimedia may be more accurately performed. In addition, in terms of the overall amount of data, there is an advantage that is possible to collect more accurate and larger amounts of data compared to data that collected through the OBDII port. Therefore, if the EDA method that we are presenting is used in vehicle related research area, it seems that results of research will be more effective and accurate for future vehicle internal network. Acknowledgments. This work was supported by the National Research Foundation of Korean (NRF) grant funded by the Korea government (MSIT) (No. 2021R1A4A2001810). This work was supported by Institute for Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korean government (MSIT) (No. 2019-0-01343, Regional strategic industry convergence security core talent training business).

References 1. Tomlinson, A., Bryans, J., Shaikh, S.A., Kalutarage, H.K.: Detection of automotive CAN cyber-attacks by identifying packet timing anomalies in time windows. IEEE (2018) 2. Song, H.M., Kim, H.R., Kim, H.K.: Intrusion detection system based on the analysis of time intervals of CAN messages for in-vehicle network. IEEE (2016) 3. Zhou, F., Li, S., Hou, X.: Development method of simulation and test system for vehicle body CAN bus based on CANoe. IEEE (2008)

Efficient CAN Dataset Collection Method for Accurate Security

107

4. Taylor, A., Japkowicz, N., Leblanc, S.: Sylvain Leblanc: frequency-based anomaly detection for the automotive CAN bus. In: WCICSS (2015) 5. Olufowobi, H., et al.: SAIDuCANT: specification-based automotive intrusion detection using controller area network (CAN) timing. IEEE Trans. Veh. Technol. 69(2), 1484–1494 (2019) 6. Qin, H., Yan, M., Ji, H.: Application of controller area network (CAN) bus anomaly detection based on time series prediction. Veh. Commun. 27, 100291 (2021) 7. Kavousi-Fard, A., et al.: An evolutionary deep learning-based anomaly detection model for securing vehicles. IEEE Trans. Intell. Transp. Syst. 22(7), 4478–4486 (2020) 8. Understanding CAN Comunication, 28 May 2021. FESCARO. https://www.fescaro.com/ko/ archives/249

A New Method for Improving Throughput Performance by Simultaneous Transmission on Full-Duplex Wireless Communication Systems Hikari Hashimoto(B) and Tetsuya Shigeyasu Graduate School of Comprehensive Scientific Research, Prefectural University of Hiroshima, Hiroshima, Japan [email protected], [email protected]

Abstract. Recently, by the rapid development of IoT (Internet of Things) [1] technologies, a bunch of devices are newly connected to a network. Hence, it is required to increase communication bandwidth for full filling quality of network service. Generally, introducing full-duplex communication [2] can increase network bandwidth, instead of half-duplex communication. In the case of wireless communications, however, existing commercial communication systems do not support the full-duplex wireless communication. Nowadays, new technology to realize full-duplex wireless communication even on a same channel has been developed despite the hardness of the elimination of self interference [3, 3–5]. For implementing a function of full-duplex communication on a wireless network, we can increase upper limit of network bandwidth, significantly. However, in order to earn privilege of full-duplex wireless communication, appropriate scheduling on simultaneous transmission is required [6–9]. For the scheduling, simultaneous transmitters must be selected carefully by considering their communication range. Therefore, this paper proposes a method to encourage multiple potential transmitters engage new simultaneous transmissions without any collision with preceding primary transmission. Characteristics of the proposal is evaluated by computer simulation.

1

Introduction

Recent years, by the development of the IoT (Internet of Things) technologies, a bunch of devices are connected to the network. We can monitor and drive a lot of things in the real spaces by both of sensors and actuators. Technology advancement in IoT field, made for the great increase of IoT devices connected to network. Then, capabilities of network bandwidth for newly added devices are highly required. It is obviously that total network bandwidth can be increased by additional communication channel which is independent from exiting channels. c The Author(s), under exclusive license to Springer Nature Switzerland AG 2022  L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 108–119, 2022. https://doi.org/10.1007/978-3-031-08819-3_11

A New Method for Improving Throughput Performance

109

In the case of wired network, it is easy to add such channel by adding new cable to the network. This is because, basically, different cable does not interfere the other cable separated physically each other. On the other hand, in the case of wireless communications, it is hard to increase a network capacity because of a difficulty of remove interferences among neighboring communication nodes in a same radio frequency. Due to the easy placement without any cabling, wireless connection to network are strongly requested to utilize the ability IoT devices. One of the strong candidate methods for increasing a channel capacity in one band, is full-duplex wireless communication, in which node pairs in same communication range transmit/receive packets, simultaneously. The full-duplex wireless communication, however, must be paid much attentions to avoid multiple receptions of packets on any one receiver on same time. Hence, we have to choose simultaneous transmitters on the basis of those physical locations, carefully. In this paper, we propose a new method to increase number of simultaneous transmissions by selecting candidate transmitters according to their interference range. The results of computer simulations confirm that own proposal effective increase the network capacity.

2

Full-Duplex Wireless Transmission

This section first describes fundamentals of the knowledge of full-duplex wireless transmission. Next, we mention the characteristics of full-duplex transmissions. 2.1

Duplex Operations on Wireless Communications

As the common knowledge, most successful wireless communication standard is IEE802.11wireless LAN [10]. Including the devices based on the standard, typical wireless devices employ not a full-duplex but a half-duplex communications. On a half-duplex transmission, only one transmitter starts its new transmission if and only if there is no other precedence transmitter in the same communication area. The reason, why the most wireless nodes do not support the full-duplex transmissions is that elimination of self-interference is hard on a same wireless channel. Figure 1 shows an example of half-duplex wireless transmission. In the figure, N i(i ≤ 2) and DATA show the node and Data packet, respectively. Since

Fig. 1. Half-duplex transmission.

110

H. Hashimoto and T. Shigeyasu

half-duplex wireless transmission and reception are not allowed, simultaneously, N2 can send its own new DATA to N1 after finishing to receive the Data from N1. Full-duplex wireless transmission allows simultaneous transmission (reception) while its receiving (transmitting) on a same channel. Figure 2 shows an example of full-duplex wireless transmission among N1 and N2. Compared to the half-duplex wireless transmission, full-duplex wireless transmission increases throughput by increasing the number of packets transmitted in a given time.

Fig. 2. Full-duplex transmission.

2.2

Characteristics of Full-Duplex Transmission

Following sections describe the transmission fashions [11] and challenges of fullduplex wireless transmission. 2.2.1 Full-duplex Transmission Fashions According to the node type of a transmitter and a receiver, full-duplex transmission can be divided into two transmission fashions: bi-directional full-duplex transmissions and relay full-duplex transmissions. The latter one can be further classified into two categories, PR-based relay full-duplex, PT-based relay full-duplex, depending on the difference in the secondary sending nodes. In the following description, a primary transmitter and a primary receiver are indicated as PT and PR, and a secondary transmitter and a secondary receiver are indicated as ST and SR, respectively. Bi-directional full-duplex transmissions can be performed if and only if the any two neighboring nodes have at least one packet destined to each other. In this fashion, PT and PR serve as SR and ST, respectively. Figure 3 shows bidirectional full-duplex communication. As shown in the figure, nodes of PT and SR, and PR and ST are allocated to the same nodes in this transmission.

Fig. 3. Bi-directional full-duplex transmission.

A New Method for Improving Throughput Performance

111

Relay full-duplex communications are the full-duplex transmissions performed by three nodes. PR-based relay full-duplex communications are performed when a PR has a transmission packet to a node other than PT. As shown in Fig. 4, although one node serves as PR and ST, roles of PT and SR are allocated to the different node, respectively. When PR does not have any transmission packet except destined for PT, only PT-based relay full-duplex transmission can be performed. In this fashion, roles of PR and ST will be allocated to the different nodes. Figure 5 shows PT-based relay full-duplex transmission. As shown in the figure, roles of PT and SR are allocated to the same node different from the nodes of PR and ST.

Fig. 4. PR-based relay full-duplex communication.

Fig. 5. PT-based relay full-duplex transmission.

2.2.2 Challenges of Full-Duplex Wireless Transmission One of the challenges of full-duplex transmission is increasing number of successful transmissions [6]. In other for full-duplex wireless transmission to be successful, at least one of the SR candidate nodes within the communication range of the PT must hold a packet destined to the appropriate node [8,9,12]. For example, in the case of the transmission shown in Fig. 3, it is necessary for PR to have a packet destined to the PT. If this condition is not fulfilled, a secondary transmission can not be performed, then, only the half-duplex wireless transmission is applicable. As a result, the nodes can not receive a benefit for full-duplex wireless communication.

3

Methods for Selecting Candidate Transmitter

In this section, we propose a new method for selecting candidate transmitter on full-duplex wireless communication. In the proposal, first, PT initiates primary transmission of a relay full-duplex communication, then, multiple candidate transmitters designated by PT, start their transmission in parallel with PT’s transmission. For designated candidate nodes, PT registers candidate node

112

H. Hashimoto and T. Shigeyasu

ID on its Data header. Here, candidate nodes will be selected from nodes sharing no collision domain, each other. In other word, no collision will be occurred even if all candidate nodes designated by PT, make their transmission. Hence, it is expected that by implementing our method, network throughput can be maximized if the candidate nodes hold packets destined to appropriate node. 3.1

Transmission Procedure

Figures 6 and 7 show the example topology and transmission sequence of our proposal on the topology. On the Fig. 7 shows, horizontal line indicate each node’s transmission/reception state. For example, top most line indicates the state of SR1. On each line, packet transmitted by itself is indicated above the line, and, packet received at the node is indicated below the line. As shown Fig. 7, ST1 (ST2) sends to SR1(SR2) the S1(S2) DATA, in parallel with PT’s primary transmission. On the Fig. 6 PR receiving the primary transmission, starts PR-based relay full-duplex transmission. Neighbors of the PT check if it is designated as candidate node on a overhearing Data packet. Node which is not registered on the header of PT’s Data, sets its state as busy, and defers its new transmission to avoid affecting the on-going transmission. 3.2

Selection Algorithm of Candidate Transmitters

As mentioned the above, PT informs candidate transmitters for giving transmission authorization. For the purpose, PT calculates the authorization node list according to its surrounding topology. The procedures of making the list are described below. 1. Pick up the nodes fulfilling the both following 2 conditions: • neighboring to PT, and • not connected to PR 2. For each node selected at the above step, calculate the influece 3. Add a node to the candidate nodes, if the node had a minimum value of influence 4. Remove the node selected at the previous step and its neighbor nodes from network, and, back to step 1 if and only if any node exist with in the transmission range of T. Otherwise, finish this selection procedure. In the step 2 on the above procedure, value of influence is a number of nodes that are 2 hops away from PT and neighbors of the candidate node. For example, if we focus the node 5 as a candidate node, the value of node 5’s influence is 2 in Fig. 8.

A New Method for Improving Throughput Performance

113

Figure 8 shows result of the application of our proposal for the example topology of Fig. 6. In this figure, nodes shown as double (black) circle allowed (deferred) their new synchronous transmission. Next, for the remained neighbors of PR, value of influence is derived. According to the derived values, node 3 having the largest value, is newly added to the authorization list. For making the result, firstly, node 2 removed from the candidate because node 2 is connected to both PT (node 0) and PR (node 1). After adding node 3, node 4 will be removed from the candidate cause it is a neighbor of node 3. These steps is repeated until all nodes of PT’s neighbor is added to the authorization list or is removed from candidate node.

Fig. 6. Example topology.

3.3

Fig. 7. Transmisson sequence.

Method for Selecting Destination

Although multiple nodes designated by PT’s transmission can start new transmission, these nodes must select its destination node from other than the PT’s neighbors. The reason for it is to avoid collisins with the primary transmission. Hence, in Fig. 8, node 1 can only send to node 9 or node 10.

4

Performance Evaluations

This section evaluates the performance of the proposed method. In this evaluation, the proposal method applied to 1) random network, 2) fixed network with one way traffic flows. For both situations, we evaluate performances in terms both of throughput and successful reception ratio.

114

4.1

H. Hashimoto and T. Shigeyasu

Random Network

Simulation parameters are shown in Table 1, we compare four duplex methods: half-duplex, full-duplex(all), full-duplex(self) and full-duplex(list). Here, in fullduplex(all), all nodes in PT’s communication range make sure to in parallel with a primary transmission. In addition, for full-duplex(self) is PR-based relay full-duplex communication will be performed in evaluations.

Fig. 8. Network topology. Table 1. Simulation parameters Parameter

Value

Transmission speed

1 Mbps

Communication range 100 m Simulation period

10.0 s

Number of terminals

100

Packet arrival process Poisson distribution

Figure 9 shows characteristics of throughput. According to the results, we can see that the proposed method effectively improved the throughput performance compared to half-duplex and full-duplex(all). However, we can not recognized advantages for full-duplex(self).

A New Method for Improving Throughput Performance

115

Fig. 9. Throughput performance on random network.

Figure 10 shows successful reception ratio. As the figure shows, result of proposed method achieves high successful reception ratio compared to fullduplex(all). Although, our proposal and full-duplex(self) achieved almost highest performance, these two methods got behind the half-duplex traffic condition. In addition, our proposal, full-duplex(list) little bit got behind the fullduplex(self). The reason is that our proposal induces collisions among transmissions belonging to the different PT. For more concretely, any PT can only avoid collisions among transmissions designated itself. Hence, the PT can not control the transmissions designated by other PT. 4.2

Fixed Network with One Way Traffic Flows

This section reports the results of performance evaluation using fixed network with one way traffic flow. The simulation specifications shown in Table 1. Figures 11, 12 and 13 show the examples of nodes of node on the each method when the case of PT and PR are node 0 and node 1, respectively. On the fixed network, all generated traffics will be transferred toward outward from node 0. This model is supposed the Data collections from sensor node by flooding on IoT network.

116

H. Hashimoto and T. Shigeyasu

Fig. 10. Successful receive ratio on random network.

Fig. 11. Half-duplex.

Fig. 12. Full-duplex(all).

Figure 14 shows characteristic of throughput. As the figure shows proposal method achieves the highest throughput. Figure 15 shows successful reception ratio. Result of proposal method achieves almost 100% successful reception ratio.

A New Method for Improving Throughput Performance

Fig. 13. Full-duplex(self).

Fig. 14. Throughput performance on fixed network.

117

118

H. Hashimoto and T. Shigeyasu

Fig. 15. Successful reception ratio on fixed network.

5

Conclusion

For realizing high quality network having enough capabilities to admit bunch of IoT devices, this paper discussed a method for increasing successful transmissions by full-duplex technologies. In our proposal, adequate candidate transmitters are carefully selected on their communication range. Simulation results confirmed that our proposed method effectively increased throughput performance. In the future, we would like to deal with the remained issues that the increase of collisions among transmitters belonging to the other PT’s full-duplex transmissions.

References 1. Al-Fuqaha, A., Guizani, M., Mohammadi, M., Aledhari, M., Ayyash, M.: Internet of Things: A Survey on Enabling Technologies, Protocols, and Applications. IEEE Commun. Surv. Tutor. 17(4), 2347–2376 (2015). https://doi.org/10.1109/COMST. 2015.2444095. Fourthquarter 2. Liao, Y., Bian, K., Song, L., Han, Z.: Full-Duplex MAC Protocol Design and Analysis. IEEE Commun. Lett. 19(7), 1185–1188 (2015). https://doi.org/10.1109/ LCOMM.2015.2424696 3. Duarte, M., Dick, C., Sabharwal, A.: Experiment-driven characterization of fullduplex wireless systems. IEEE Trans. Wireless Commun. 11(12), 4296–4307 (2012). https://doi.org/10.1109/TWC.2012.102612.111278

A New Method for Improving Throughput Performance

119

4. Choi, J.I., Jain, M., Srinivasan, K., Levis, P., Katti, S.: Achieving single channel, full duplex wireless communication. In: Proceedings of the 2010 ACM MobiCom, pp. 1–12 (2010) 5. Sabharwal, A., Schniter, P., Guo, D., Bliss, D.W., Rangarajan, S., Wichman, R.: In-band full-duplex wireless: challenges and opportunities. IEEE J. Sel. Areas Commun. 32(9), 1637–1652 (2014). https://doi.org/10.1109/JSAC.2014.2330193 6. Achaleshwar, S., Gaurav, P., Ashutosh, S.: Pushing the limits of full-duplex: design and realtime implementation. In: Rice University Technical Report TREE1104 (2011) 7. Jainy, M., et al.: Practical, real-time, full duplex wireless. In: Proceedings of the ACM 17th Annual International Conference on Mobile Computing and Networking, ACM MobiCom 2011 (2011) 8. Tamaki, K., Raptino, H.A., Sugiyama, Y., Bandai, M., Saruwatari, S., Watanabe, T.: Full duplex media access control for wireless multi-hop networks. In: 2013 IEEE 77th Vehicular Technology Conference (VTC Spring), pp. 1–5 (2013). https://doi. org/10.1109/VTCSpring.2013.6692573 9. Cheng, W., Zhang, X., Zhang, H.: RTS/FCTS mechanism based full-duplex MAC protocol for wireless networks. In: 2013 IEEE Global Communications Conference (GLOBECOM), pp. 5017–5022 (2013). https://doi.org/10.1109/GLOCOMW. 2013.6855746 10. Bianchi, G.: Performance analysis of the IEEE 802.11 distributed coordination function. IEEE J. Sel. Areas Commun. 18(3), 535–547 (2000). https://doi.org/10. 1109/49.840210 11. Goyal, S., Liu, P., Gurbuz, O., Erkip, E., Panwar, S.: A distributed MAC protocol for full duplex radio. In: 2013 Asilomar Conference on Signals, Systems and Computers, pp. 788–792 (2013). https://doi.org/10.1109/ACSSC.2013.6810393 12. Singh, N., Gunawardena, D., Proutiere, A., Radunovi, B., Balan, H.V., Key, P.: Efficient and fair MAC for wireless networks with self-interference cancellation. In: 2011 International Symposium of Modeling and Optimization of Mobile, Ad Hoc, and Wireless Networks, pp. 94–101 (2011). https://doi.org/10.1109/WIOPT.2011. 5930070

Intelligent Helmet Supporting Visually Impaired People Using Obstacle Detection and Communication Techniques Linh Thuy Thi Pham1,4 , Khoa Thanh Nguyen2 , Duyen Thuy Dao2 , Hai Thanh Nguyen1(B) , Huong Hoang Luong3 , and Nhan Trong Pham Van3 1

4

Can Tho University, Can Tho, Vietnam [email protected], [email protected] 2 An Khanh High School, Can Tho, Vietnam dtduyen2020 [email protected] 3 FPT University, Can Tho, Viet Nam FPT High School, FPT University, Can Tho, Vietnam

Abstract. Blindness is also known as a symptom of loss of visual perception. People who are completely blind cannot perceive light and dark cannot see what is around them. This study focuses on designing a guide helmet for the blind with intelligent, superficial, affordable, friendly characteristics and deploying it to improve the mobility of both the blind and the visually impaired. The proposed method includes a wearable device, including headgear, to help blind people navigate safely alone and avoid any obstacles encountered, whether fixed or movable, to prevent accidents. The main component of this system is an ultrasonic sensor that scans obstacles through reflected waves. The reflected signals received from the barrier objects are used as input to the microcontroller. The microcontroller is then used to determine the direction and distance of objects around the blind. In addition, the system integrates a subscriber identity module card (SIM800A) and an Imou camera to protect the blind and communicate with family members in emergency cases. In addition, the family members can stream video from the camera to know whether the blind person has gotten in danger or not. The experiments were done in Federation of the Blind in Can Tho city, Vietnam, to evaluate the efficiencies of the proposed method and aim to compare several products. Keywords: Intelligent helmet detection

1

· Visually impaired people · Obstacles

Introduction

At least 2.2 billion people in the world have a near or distance vision impairment based on the report by World Health Organisation (WHO)1 . The majority 1

https://www.who.int/news-room/fact-sheets/detail/blindness-and-visualimpairment, accessed on 11 March 2022.

c The Author(s), under exclusive license to Springer Nature Switzerland AG 2022  L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 120–131, 2022. https://doi.org/10.1007/978-3-031-08819-3_12

Intelligent Helmet Supporting Visually Impaired People

121

of people with vision impairment and blindness are over the age of 50 years; however, vision loss can affect people of all ages [1]. In addition, the prevalence of distance vision impairment in low- and middle-income regions is four times higher than in high-income regions. For example, Vietnam has more than 2 million visually impaired people2 . Blindness is a lack of visual perception due to physiological or neurological factors. Partial blindness represents the lack of integration in the growth of the optic nerve or visual center of the eye, and total blindness is the entire absence of the visible light perception [2]. Blind people cannot see but can only hear with their ears and perceive things around them with the rest of their senses. The inability to see also causes difficulties when moving on the road, so it is easy to collide with objects. Walking sticks are the most frequently used mobility aids for blind people to move independently and safely. However, the cane cannot detect obstacles in the upper part of the body. Several studies related to smart devices to assist the blind have been proposed to ensure the safety of the blind. We can mention it as an intelligent guide white cane used to detect obstacles under their feet. However, this tool has some limitations, including the length of the stick, recognizing obstacles, and the difficulty of keeping it in public places. Research reports on mobility assistance systems for the blind show that obstacle detection technology can rely on infrared, laser, image, and ultrasonic sensors. However, infrared and laser sensors have the limitation that they cannot accurately detect objects with angular shapes due to reflections, while image sensor cameras can only be used during the day. In contrast, ultrasonic sensors are not affected by the temperature of the vehicle’s engine, do not affect humans, and can be used both during the day and at night, especially at a low cost. Many studies focus on navigation for the blind based on Global Positioning System (GPS) [3–5]. The GPS receiver receives signal information from the satellite and determines the user’s location based on trigonometry. GPS can accurately determine the user’s location and display the results on the electronic device. The advantages of GPS include accuracy and ease to use, working well in all climate conditions, free, global coverage, easy integration right into the phone, and convenience for users. While the GPS is generally perfect, several factors can interfere with the signal, cause loss of transmission time, and reduce its accuracy. In addition, it may run out of battery during use, and users need to equip a backup power source during long trips. Our research proposes an obstacle warning device for the visually impaired with an ultrasonic sensor, emergency calls to their loved ones when they are in danger through sim800A, and their loved ones can know to position as well as observe live images from the visually impaired moving outside through the camera, bringing peace of mind to the family as well as the life of the blind. Moreover, it can help visually impaired people easily integrate into the community and feel secure when on the road. 2

http://t5g.org.vn/viet-nam-co-khoang-2-trieu-nguoi-mu-va-thi-luc-kem, on 11 March 2022.

accessed

122

2

L. T. T. Pham et al.

Related Work

A simple, cheap, configurable, easy-to-handle electronic guidance system proposed to provide constructive assistance and assistance to blind and visually impaired people studied by [6]. The system was designed, implemented, tested, and verified with 93% accuracy in detecting different shapes, materials, and distances. Furthermore, it can scan the left, right, and front areas of a blind person regardless of its height or depth, which is done by IR infrared sensor. An intelligent Electronic Traveling Aid (ETA) called BlinDar was suggested in [7]. BlinDar was a lightweight, low power consumption, and cost-effective helmet for blind people. Ultrasonic sensors have been used to detect obstacles and potholes within 2 m. The GPS and ESP8266 Wi-Fi modules were used to share the location with the cloud. In addition, the MQ2 gas sensor is used to detect the flame in the path, and an RF Tx/Rx module to find it when it is misplaced quickly. The research in [8] was an assistive device for blind people that can move independently to where they want to go using a GPS receiver, and a loved one or a remote operator can respond immediately Instantly and provide assistance or remote guidance in case of emergency by monitoring video stream. The device records the location name requested by the user in the Voice module, providing navigation to the location request using the GPS receiver by receiving the GPS signal containing the latitude and longitude values from the GPS satellite and the GPS receiver. With location name announcement from Voice module, monitor user from computer video and help when needed. Another work in the smart home in [9] was created based on an IoT-based system consisting of sensors and antennas to receive and transmit signals. Bluetooth devices are installed in every room, and blind people wear watches. With the help of multiple ultrasonic sensors, the watch detects static or dynamic obstacles and provides audio signals to the user for navigation. When the user enters a specific room, a Bluetooth device connects to the watch and notifies the user of its location. The recommendation system gives blind people the ability to navigate and avoid obstacles. The authors in [10] presented the method based on Object Recognition Cap (ORCap) using a computer vision-based approach. It is a wearable system with a simple architecture that uses object detection to convert real-time video captured from the camera module into text information using a Raspberry Pi board. This device can quickly detect objects of various classes with minimal computational cost. ORCap can make navigating the path easy as it is a real-time system, and it monitors the environment descriptively, providing acoustic information of the objects present and their position to the user. In [11], researchers attached a Radar device with a precise millimeter-level signal to a white walking stick. This device can detect obstacles and distinguish humans from usual obstacles to Anticipate potential hazards. It works by identifying the small displacements of the ribcage caused by physiological activity that is the main sign of a human presence. Smart White Cane [12] is recommended as a Sophisticated and Economic Walking Aid. Their proposed stick is designed to detect obstacles to help blind people navigate comfortably. Their device comprises an ATMega328PU

Intelligent Helmet Supporting Visually Impaired People

123

microcontroller, four ultrasonic sensor modules HC-SR04, audio IC-APR33a3, a vibration motor, headphones, and batteries with Audio feedback. Their method alerts users with pre-recorded audio messages and haptic feedback in the form of vibrations. The cane can detect holes, potholes, falls, stairs (up and down), low and knee obstacles, and even obstacles higher than the waist. The battery compartment is hidden in the bar, thus reducing the risk of damaging the circuit and making the device less bulky. In addition, the system offers an ON/OFF switch, vibration feedback, and an audio jack on the handlebar. However, the system has no locating method to find the user’s location, cannot detect oncoming vehicles and slippery floors, and has no fire or smoke alarms. The above studies show that devices to assist the visually impaired focus on detecting obstacles with white canes for navigation and positioning via GPS, but there is little research on warning when meeting blind people when they are blind with dangerous incidents. Therefore, it is perfect for designing a helmet to assist the visually impaired in avoiding obstacles, calling their loved ones in case of an accident, or observing them through live video streaming on phones or computers completely relevant and urgent.

3 3.1

Methods Requirements

The helmet does not interfere with regular sensory habits for walking, such as requiring the user to wear gloves or interfering with hearing environmental sounds. The helmet should be a small weight (the lighter, the better). The total weight should not exceed 1 kg, helping the user use it for a long time. Each battery charge for about 5 h allows about one week of use before charging the battery if the average person travels about 2 h a day. The helmet should own high detection accuracy (at least 90% or more), and the time for detecting must be 0.3 s or less within a radius of 100 cm. When there is an unexpected problem, the visually impaired can contact emergency relatives through default calls and messages for help with just a simple press without having a phone attached. Relatives can see where the blind person has gone at any time or review the memory card’s recording history to know where they have been in danger. The price is as low as possible to suit the economic conditions of the majority of blind people in Vietnam. The sensor must detect obstacles from waist level upwards to the user’s head at a distance of 1 to 4 m so that the user has time to react and not be confused with distant objects. The helmet can be used after 10 min of instruction. Therefore, all visually impaired people can use it easily through instruction, suitable for both children and adults, blind people due to advanced age. 3.2

Components for the Designed Helmet

SIM800A. SIM800A is a quad-band GSM/GPRS module that works on frequencies GSM 850 MHz, EGSM 900 MHz, DCS 1800 MHz, and PCS 1900 MHz.

124

L. T. T. Pham et al.

SIM800A features GPRS multi-slot class 12/class10 (optional) and supports the GPRS coding schemes CS-1, CS-2, CS-3 and CS-4. With a tiny configuration of 24 * 24 * 3 mm, SIM800A can meet almost all the space requirements in users’ applications, such as M2M, smartphones, PDA, and other mobile devices. SIM800A is designed with a power-saving technique so that the current consumption is as low as 1.2 mA in sleep mode. SIM800 integrates TCP/IP protocol and extended TCP/IP AT commands useful for data transfer applications.

Fig. 1. SIM 800A-GSM modem

Ultrasonic Sensors. Ultrasonic sensor HY-SRF 05 is used to measure the distance from the wearer to the obstacle. Specifications include Voltage 5 VDC, the maximum angle of 15◦ , and distance of 2 cm–450 cm. The sensor can emit ultrasonic waves with a specific opening angle. At that time, if it detects an obstacle within its scanning range, the ultrasonic wave will reflect. Therefore, we can measure the distance by calculating the time from the time the ultrasonic wave is emitted to the time it is received and then combining it with the ultrasonic speed (343 m/s) to know the distance the wave has traveled. The advantage of ultrasonic sensors is fast processing and relatively accurate results. However, its shortcomings can be mentioned as the ultrasonic sensor only recognizes obstacles when the sensor’s scanning plane crosses the obstacle, so it does not detect small, low, close obstacles ground. Moreover, due to ultrasonic waves and their reflection to calculate the distance and detect obstacles, some errors are difficult to overcome, such as repeat errors, the Forecasting phenomenon, and the Crosstalk cross-reading phenomenon. The forecasting phenomenon is the phenomenon of the wrong angle reflection of the sensor. According to the TOF principle, the ultrasonic sensor must be directed perpendicular to the obstacle’s surface to measure the correct distance. However, obstacles are not always smooth, so the reflected ray may not

Intelligent Helmet Supporting Visually Impaired People

125

correspond to the angle of incidence. As a result, the results of the ultrasonic sensor readings are skewed due to the large opening angle of the ultrasonic sensor. Finally, the crosstalk phenomenon is a phenomenon where ultrasonic sensors record signals reflected either directly from other ultrasonic sensors or after the process of ultrasonic waves traveling and reflected through surfaces unexpectedly back to the sensor. 3.3

Design for the Helmet

The hardware design of the helmet is as shown in Fig. 2 including a half helmet with a diameter of 58cm and a weight of 800 g. The internal circuit board design includes two vibration motors, speakers, speakers, and three ultrasonic sensors mounted on the left, right, and front sides. The emergency call button is located at the top. In addition, we installed an Imou 1080p full HD camera3 and Imou Life app4 for camera monitoring. The power supply includes three batteries (3.7 V–6800 mAh). Figure 4 shows the schematic design of the system, including the 2-channel Opto Relay Module to select the High/Low trigger level to turn on and off the led lights and the vibrating motor. The Arduino Nano board acts as a central processing unit, receiving signals from SIM800A, processing signals, and outputting signals (Digital, PWM) to implementing devices (Module Relay, Motor Vibration). The power supply for the active cone includes three batteries of 3.7 V– 6800 mAh. The Arduino mini vibrating motor driver circuit checks the input of

Fig. 2. The design for a smart helmet 3 4

https://www.imoulife.com/vn/product/detail/Cue2, accessed on 14 March 2022. https://play.google.com/store/apps/details?id=com.mm.android.smartlifeiot& hl=vi&gl=US, accessed on 14 March 2022.

126

L. T. T. Pham et al.

Fig. 3. Circuit diagram of cone

the vibrating motor. If it is high, the motor will vibrate. The HY-SRF05 ultrasonic sensor emits an ultrasonic wave and then measures the time from emission to return of the wave to measure the distance from the current position to the object to be measured. The XL4015 DC voltage reducer circuit has a current regulator to ensure safety when used. In overcurrent, the circuit will warn through the led light and automatically disconnect. The SIM800A module will read the status of the emergency button. If the button is turned on, the SIM will send messages and calls to the preset relatives. The switch is used to turn the helmet on and off. The Imuo 5 V–1 A camera, connected to the source via relay to turn on and off, can be viewed remotely via phone and supports a global wireless connection. The LED lights up when it encounters an obstacle. The speaker will emit a sound when receiving a signal from the HY-SRF05 ultrasonic sensor so that the user knows there is an obstacle. The schematic diagram of the proposed system is shown in Fig. 3. When the switch is turned on, the Arduino Module will be initialized. The data inputs will be activated. The ultrasonic sensors will be input. Continuously checking the signal to see an obstacle will activate the vibration motor when the distance value exceeds 20 cm. Distances greater than 500 cm are counted as out of coverage area. SIM800A module can check and send a message according to the predefined syntax for the preset phone number if the push button is activated.

Intelligent Helmet Supporting Visually Impaired People

127

Fig. 4. The workflow for the helmet.

4 4.1

Experiments and Evaluation Operating Principles

First, attach the battery and turn on the power switch like Fig. 5. Then, the helmet will send a text message to notify the loved one that “The helmet is ready”. Obstacle Avoidance Function. The product is integrated with three ultrasonic sensors front, left, and right. When a blind person moves 50 cm from an obstacle from the left and right, the vibrating motor will operate, signaling an obstacle. If an obstacle is in front of you, the buzzer will signal, and the vibrating motor will work. The horn and vibrating motor will stop if the obstacle is far from them.

128

L. T. T. Pham et al.

Fig. 5. The location of the helmet’s battery, power switch and emergency buttons

Contact Relatives, Family Members. When the blind have a dangerous incident, they can press the emergency call button on the top of the hat. When this button is pressed, a call will be made to the family member after 3 s from the user’s preset number in SIM800A. At the same time, a text message will be sent to the phone number with the content “Help me, Please!” shown in the figure. The Function of Viewing the Visually Impaired via Camera. Family members or Relatives can download the Camera Imou Life app5 to their phones. Imou has integrated applications on both Android and iOS operating systems. Then, scan the device code with the camera to connect. Then, we can observe the recorded video whenever we need like Fig. 6. In addition to viewing and tracking functions, the camera also integrates several additional features such as 2-way talk reviewing recorded history. 4.2

Evaluation

Design. With the helmet’s design, the product is compact, simple, and not fussy. The product uses three batteries, so it is light for users instead of ordinary batteries. The circuit and power supply are designed inside the helmet, so it looks like a regular helmet. In addition, the circuit can be protected against environmental factors safe for users, protecting users. Obstacle Detection. With the design of three ultrasonic sensors, HY-SRF05 is designed in the front and left, and right of the helmet to help blind people receive warnings about dangers from many directions with a distance of 20 cm– 200 cm. The advantages of ultrasonic sensors are fast processing and relatively accurate results. So we tested the obstacle between 20 cm and 200 cm with 99% accuracy for 100 tests. The ultrasonic sensor recognizes obstacles >200 cm to 450 cm with reduced accuracy, but that distance will not warn the blind because 5

https://www.imoulife.com/vn/support/download/app.

Intelligent Helmet Supporting Visually Impaired People

129

Fig. 6. Imou software interface to observe the blind’s position and screen of calls and messages about loved ones

for blind people and obstacles fixed with a distance of distance >200 cm are considered safe. The disadvantage of the ultrasonic sensor is that it uses ultrasonic waves and its reflection to calculate the distance and detect obstacles, so there are some problematic cases to overcome, such as repetition error, Forecasting phenomenon, and reading phenomenon Crosstalk. Therefore, the product still suffers from signal interference that makes the motor unable to stop vibrating, or the buzzer continuously howls when an obstacle is detected (probability of signal interference 10% after tests). Moreover, the system has not yet warned against fast-moving obstacles to blind people and has not yet classified the materials of obstacles such as wood, glass, and walls. Observation by Camera. An evaluation of the observation function through the camera with a good overview with a 2.8 mm fixed lens, diagonal viewing angle to 131◦ has experimented. The helmet provides sharp observation images with Full HD (1080p) resolution. Observe in low light with 10 m infrared vision, including 2-way talk support for convenient camera chats and good anti-interference ability. Information can be stored via a MicroSD card (up to 256 GB), NVR recorder, or Cloud server. In addition, relatives can review the movement history of the visually impaired when installing the manufacturer’s Imou app. Contact Family Members. We evaluated the function to call relatives in case of emergency. The helmet uses SIM800A to receive signals from the emergency button when the visually impaired have an accident. After 2–3 s of pressing the

130

L. T. T. Pham et al.

emergency signal button, relatives can receive a call from the hat’s phone number and a text message. Power Consumption. The helmet uses 3 Cells Ultrafire 18650 lithium battery 3.7 V–6800 mAh. The current consumption of each ultrasonic sensor is 40 mA, Arduino Nano board 40 mA, camera 1000 mA, Sim800A 1000 mA, and some other materials ranging from 5 mA to 40 mA. Therefore, if the camera is not turned on but only used to avoid obstacles, the helmet can work for 85 h (equivalent to 3.5 days). If all features are turned on, including obstacle avoidance, camera, and SOS warning, it can work continuously for 9 h. Enough time if blind people do outdoor activities. Reusable rechargeable battery to save users. 4.3

Comparison with Other Devices

Table 1. Comparison between the proposed helmet and others Products

Power

Range

Response Alerts Portable

Ours

Medium Mid-Range Fast

Yes

Portable

Smart White Cane [12] Medium Mid-range

Fast

No

Portable

BlinDar [7]

Fast

No

Portable

Medium Mid-range

Table 1 presents the comparison between our helmet and considered others. First, we compared the consumption time without recharging. The criteria are as follows: power consumption of 0–0.5 W is considered low power, 0.5–1 W is medium consumption, and higher than 1 W is high consumption. The second parameter is the operating range. An instrument that can find obstacles within 0–2 m can be considered low range, 2–4 m as mid-range, while higher than 4 m is considered high range. The third parameter is response time. The system recognizes and responds 0–100 ms is considered fast. An average of 100–200 ms and higher than 200 ms is slow. The fourth parameter is to notify loved ones when in danger and their location. Portability is another important parameter of the system. The subject can wear and use a system for long periods is considered a portable system; otherwise, it is considered unmovable. Finally, the ease of use of the system is considered another parameter. The helmet was evaluated and trial used in Federation of the Blind in Can Tho city, Vietnam.

5

Conclusion

This article focuses on building a simple, cheap system proposed to assist the blind and visually impaired in indoor and outdoor navigation. Through ultrasonic sensors, the system is designed, implemented, tested, and verified accurately to detect obstacles with different distances and directions within 2 m.

Intelligent Helmet Supporting Visually Impaired People

131

It can scan the left, right, and front areas of the blind. In addition, the system achieves 100% accuracy in calls from emergency buttons to loved ones via sim800A. From pressing the emergency button to displaying the call, the maximum time is about 2–3 s. At the same time, relatives can observe the blind through the video stream of Imou software. The system is so simple that it does not require special manual training. In the future, focus on testing more obstacle avoidance with different materials and shapes.

References 1. Blindness and vision impairment. https://www.who.int/news-room/fact-sheets/ detail/blindness-and-visual-impairment 2. Al-Fahoum, A.S., Al-Hmoud, H.B., Al-Fraihat, A.A.: A smart infrared microcontroller-based blind guidance system. Active and Passive Electronic Components 2013 (2013) 3. Hapsari, G.I., Mutiara, G.A., Kusumah, D.T.: Smart cane location guide for blind using GPS. In: 2017 5th International Conference on Information and Communication Technology (ICoIC7), pp. 1–6. IEEE (2017) 4. Baranski, P., Polanczyk, M., Strumillo, P.: A remote guidance system for the blind. In: The 12th IEEE International Conference on e-Health Networking, Applications and Services, pp. 386–390 (2010) 5. Dhod, R., Singh, G., Singh, G., Kaur, M.: Low cost GPS and GSM based navigational aid for visually impaired people. Wireless Pers. Commun. 92(4), 1575–1589 (2017) 6. Al-Fahoum, A.S., Al-Hmoud, H.B., Al-Fraihat, A.A.: A smart infrared microcontroller-based blind guidance system. Active and Passive Electronic Components 2013, 1–7 (2013). https://doi.org/10.1155/2013/726480 7. Saquib, Z., Murari, V., Bhargav, S.N.: Blindar: an invisible eye for the blind people making life easy for the blind with internet of things (IoT). In: 2017 2nd IEEE International Conference on Recent Trends in Electronics, Information Communication Technology (RTEICT), pp. 71–75 (2017) 8. Chandana, K., Hemantha, G.: Navigation for the blind using GPS along with portable camera based real time monitoring. SSRG Int. J. Electron. Commun. Eng 1, 46–50 (2014) 9. Tayyaba, S., Ashraf, M.W., Alquthami, T., Ahmad, Z., Manzoor, S.: Fuzzy-based approach using IoT devices for smart home to assist blind people for navigation. Sensors 20(13), 3674 (2020). https://doi.org/10.3390/s20133674 10. Roy, S., Gharge, S., Jasoriya, R., Agrawal, P., Jounjalkar, I.: ORCap: Object recognition cap (a navigation system for the blind). In: 2020 IEEE International Conference for Innovation in Technology (INOCON). IEEE, November 2020. https:// doi.org/10.1109/inocon50539.2020.9298310 11. Cardillo, E., Li, C., Caddemi, A.: Millimeter-wave radar cane: a blind people aid with moving human recognition capabilities. In: IEEE Journal of Electromagnetics, RF and Microwaves in Medicine and Biology pp. 1–8 (2021). https://doi.org/10. 1109/jerm.2021.3117129 12. Sheth, R., Rajandekar, S., Laddha, S., Chaudhari, R.: Smart white cane-an elegant and economic walking aid. Am. J. Eng. Res. 3(10), 84–89 (2014)

An Evacuation Support System for Promoting Distributed Evacuation in Evacuation Centers Tomoyuki Ishida(B) and Ryosuke Nitama(B) Fukuoka Institute of Technology, Fukuoka, Fukuoka 811-0295, Japan [email protected], [email protected]

Abstract. Japan is majorly affected by various natural disasters, which severely damage the country yearly. Generally, residents are forced to evacuate to nearby evacuation centers when a large-scale natural disaster occurs. However, due to the pandemic, an increasing number of residents hesitate to evacuate to these evacuation centers due to the risk of being infected with the new variants of the coronavirus. Therefore, we developed an evacuation support system to promote distributed evacuation to evacuation centers. This system visualizes the real-time congestion status of neighboring evacuation centers on Web-GIS. This allows residents to select and evacuate to uncrowded evacuation centers.

1 Introduction Japan is known to be affected by weather-related and large-scale wide-area disasters yearly. It is a major natural disaster-prone country where typhoons, heavy rains, heavy snowfalls, floods, landslides, earthquakes, tsunamis, and volcanic eruptions can occur due to natural conditions, such as location, topography, geology, and weather [1]. Currently, many local governments manually manage the safety information of residents in evacuation centers during a natural disaster. Such an analog-type management method increases human and time costs for operating evacuation centers. Therefore, this study constructs an evacuation support system consisting of an evacuees safety registration system, evacuation center management system, and evacuation center congestion status visualization system. By constructing these three systems, evacuees at each evacuation center can be accurately and promptly managed. Additionally, disaster response headquarters can distribute relief supplies using the actual conditions of the evacuation centers by grasping the situation of each evacuation center. Furthermore, residents can select a vacant evacuation center by visualizing their corresponding real-time congestion statuses. This will promote distributed evacuation and achieve both “safety of residents” and “reduced risk of infectious diseases.”

2 System Architecture Figure 1 shows the system architecture of this system, which consists of the system architectures of an administrator registration management server, evacuation center management application server, congestion status visualization application server, evacuees registration application server, and database server. Furthermore, the database server consists of the evacuation center, evacuees, and administrator information storages. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 132–139, 2022. https://doi.org/10.1007/978-3-031-08819-3_13

An Evacuation Support System for Promoting Distributed Evacuation

133

Fig. 1. Evacuation support system architecture.

2.1 Administrator Registration Management Server The administrator registration management server is responsible for administrator The administrator registration management server is responsible for administrator registration and login functions to restrict access to the evacuation center management application server. It consists of a user interface, an administrator registration manager, and a network interface. Upon login, the evacuation center management application server can be accessed. The administrator registration manager provides administrator users with administrator information registration, deletion, login, and logout functions using the user interface. 2.2 Evacuation Center Management Application Server The evacuation center management application server manages evacuation centers and is operated only by the administrator logged in as the administrator registration manager. It consists of a user interface, evacuation center registration manager, evacuees information management manager, evacuation center information visualization manager, and a network interface. The evacuation center registration manager provides functions for registering, editing, and deleting evacuation center information. Also, the evacuees’ information management manager provides functions for registering, editing, and deleting evacuees’ information. It provides an evacuees search function using the name, gender, age, etc. The evacuation center information visualization manager visualizes the number of evacuees at each evacuation center in a table with an open street map [2] using the evacuees’ registration application server.

134

T. Ishida and R. Nitama

2.3 Congestion Status Visualization Application Server The congestion status visualization application server is used by residents considering evacuating to evacuation centers. It consists of a user interface, evacuees map visualization manager, evacuees table visualization manager, and a network interface. The evacuees’ map visualization manager visualizes the evacuation center information registered in the registration manager of the management application server and the evacuees’ information registered in the count manager of the evacuees’ registration application server on the open street map. The evacuees’ table visualization manager visualizes both the evacuation center and the evacuees’ information in a table format. 2.4 Evacuees Registration Application Server The evacuees’ registration application server is used for evacuees arriving at evacuation centers to register their safety information. It consists of a user interface, evacuees update manager, evacuees count manager, and a network interface. The evacuees’ update manager registers their information and updates the registered information. The count manager counts the number of evacuees evacuated for each evacuation center. 2.5 Database Server The evacuation center information storage stores information on evacuation centers and the number of evacuees for each evacuation center using both the evacuation center management and evacuees registration application servers. The evacuees’ information storage stores the safety information of evacuees using the evacuees’ registration application server. Also, the administrator information storage stores administrator user information using the administrator registration management server. The evacuation center information storage displays evacuation centers by visualizing their congestion status, searching for safety information by the administrator, and inputting, editing, and deleting evacuation center information. The evacuation center information storage table consists of an ID, evacuation center name, latitude, longitude, capacity for each evacuation center, current number of evacuees, information registration date and time, and information update date and time. Table 1 shows the structure of the evacuation center information storage. The evacuees’ information storage registers the safety information entered by the user and searches the safety information by the administrator. It consists of an ID, evacuation center ID, evacuees’ name, age, gender, current status, and information registration date and time. Table 2 shows the structure of the evacuees’ information storage. The administrator information storage is used for login and logout and managing administrator information. It consists of an ID, login email, login password, and information registration date and time. Table 3 shows the structure of the administrator information storage.

An Evacuation Support System for Promoting Distributed Evacuation

135

Table 1. Structure of the evacuation center information storage. Item name

Data type

Remarks

Id

Bigint

Data identification number

Center_name

String

Evacuation center name

Lat

Float

Latitude

Lon

Float

Longitude

Capacity

Bigint

Seating capacity

Count

Bigint

Current number of evacuees

Created_at

Timestamp

Information registration date and time

Updated_at

Timestamp

Information update date and time

Table 2. Structure of the evacuees’ information storage. Item name

Data type

Remarks

Id

Bigint

Data identification number

Center _id

Bigint

Evacuation center ID

Name

String

Evacuee name

Age

Bigint

Age

Sex

Bigint

Sex

Remarks

String

Current status

Created_at

Timestamp

Information registration date and time

Table 3. Structure of the administrator information storage. Item name

Data type

Remarks

Id

Bigint

Data identification number

Email

String

Login email

Password

String

Login password

Created_at

Timestamp

Information registration date and time

3 Prototype System In this paper, we built a Web application with a flexible design that is independent of the OS or terminal environment of the user. This system guides evacuees to the safety information registration website using various information reading means, such as BLE beacon, NFC tag, and QR code. For example, when an evacuee scans the QR code registered in the URL with a mobile terminal, a notification guide to the safety

136

T. Ishida and R. Nitama

information registration website is displayed on the mobile terminal (Fig. 2). The user can access the safety information registration website by tapping the notification.

Fig. 2. Notification displayed on the mobile device by scanning the QR code.

Figure 3 shows the safety information registration screen when accessing the safety information registration website using a mobile terminal.

An Evacuation Support System for Promoting Distributed Evacuation

137

Fig. 3. Safety information registration screen on a mobile terminal.

The default page of the evacuation center management system is the center list screen. Figure 4 shows the evacuation center list screen. Here, the congestion status of evacuation centers was visualized in a table format. When the administrator selects the “Delete” button on the evacuation center list screen in Fig. 4, a confirmation pop-up for deleting the evacuation center is displayed, as shown in Fig. 5. If the user selects “OK,” the evacuation center is deleted. Upon deletion, a navigation bar is displayed at the top of the page to inform the administrator of the evacuation center deletion, as shown in Fig. 6.

138

T. Ishida and R. Nitama

Fig. 4. Evacuation center list screen.

Fig. 5. Confirmation pop-up when deleting an evacuation center.

Fig. 6. Navigation bar for evacuation center deletion notification.

An Evacuation Support System for Promoting Distributed Evacuation

139

4 Conclusion In this paper, we described the development of an evacuation support system for distributed evacuation by visualizing the congestion status of evacuation centers. Furthermore, we constructed the evacuees’ safety registration system that registers their information on the safety information registration website using a BLE beacon, NFC tag, and QR code. Additionally, we constructed the evacuation center management system that manages the evacuees’ information for each evacuation center registered from the safety registration system. Furthermore, we constructed an evacuation center congestion status visualization system that visualizes the congestion status of evacuation centers in real-time from the number of evacuees in each evacuation center integrated into the evacuation center management system.

References 1. Cabinet Office: White Paper on Disaster Management (2006). http://www.bousai.go.jp/kai girep/hakusho/h18/bousai2006/html/honmon/index.htm. Accessed April 2022 2. OpenStreetMap: OpenStreetMap API. https://wiki.openstreetmap.org/wiki/API. Accessed April 2022

Evaluation on Noise Reduction in Subtitle Generator for Videos Hai Thanh Nguyen, Tan Nguyen Lam Thanh, Tai Le Ngoc, Anh Duy Le, and Dien Thanh Tran(B) Can Tho University, Can Tho, Vietnam {nthai.cit,thanhdien}@ctu.edu.vn

Abstract. Currently, watching movies and videos on the internet serves all needs such as learning, entertainment, and research. The application of artificial intelligence in translation and speech recognition is also discussed. They are also developing in many research directions, such as Speech-to-Text recognition applications based on specific audio files. However, studies often focus on improving the processing speed and the accuracy of words converted into text inside the audio file but have not focused on clarifying the voice inside the audio file to facilitate easy and accurate identification. Like no tool can automatically create subtitles for videos for free, but only manually create subtitles based on timestamps and adding subtitles, which is quite time-consuming for long movies or videos. Therefore, this study proposes a new approach by combining audio processing for noise reduction, noise removal, and audio-to-text recognition to create a tool to generate subtitles automatically with high accuracy. The study results are only experimental to create a research direction that can be developed and implemented into viable applications for creating subtitles for videos without having to do it manually and with an accuracy of about 80%. Keywords: Internet services

1

· Audio · Noise reduction

Introduction

Have you watched a foreign movie without subtitles? Or have you ever followed a tutorial downloaded online in English and tried to listen again but still could not hear and understand the content? Currently, some people download movies and videos from the internet to watch offline. Still, sometimes we cannot find the subtitle file for a particular movie or video that we have downloaded. Recently, a solution is to upload the video that we need to have subtitles on to Youtube and then wait for Youtube to create the subtitles. Although Youtube recognizes the audio to develop subtitles quite well, this method is quite cumbersome. Moreover, it does not optimize the steps. It takes much time to upload the video, wait for the subtitles to be created, and watch it again. In addition, sometimes subtitles are not correct with the words, or the voices that need to be subtitled cannot be recognized. c The Author(s), under exclusive license to Springer Nature Switzerland AG 2022  L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 140–150, 2022. https://doi.org/10.1007/978-3-031-08819-3_14

Evaluation on Noise Reduction in Subtitle Generator for Videos

141

Therefore, the application automatically creates subtitles for videos plays an essential role. It helps us can understand the video content quickly and accurately. Consequently, it helps a lot in learning and gets more helpful information for users. In addition to the problem of automatically generating subtitles for videos, this study also focuses on reducing the noise of the purpose audio files so that through Speech-to-Text recognition, it will produce better quality when there is no noise filtering. The main goal of this study is to combine the noise reduction algorithm based on the “Spectral Gating” method and the Speech-to-Text API of Google Cloud to help users create subtitles for videos in multiple languages. Of course, we want to see different languages, especially movie enthusiasts, but the language difference is that we cannot understand the content for people with hearing impairment.

2

Related Work

Speech-to-text applications have been used in [4,9,10,12,14,15] speech-to-text applications that support many different things such as research, learning. With a vast of applications, subtitles generators have attracted many researchers. The work in [15] deployed the Microsoft translation service and evaluated their experiences with such a tool. Google’s Speech-to-Text API is used in [6]. It is also mentioned in the application of voice search tasks when using the user’s location as a contextual cue. The authors in [4] developed Tagalog Video Clips, which was expected to create an English subtitle from Tagalog video clips. The study in [8] investigated some methods of readability recommendations in W3C and ISO/IEC/ITU and indicated that novels feature immersive subtitle presentations. An automatic subtitle generation and semantic video summarization technique are also mentioned in [1]. The proposed method generates the subtitle for videos with/without subtitles using speech recognition and then applies NLPbased text summarization algorithms to the subtitles to show the positive result. An end-to-end system for subtitle text extraction from movie videos in [5] by using optical character recognition (OCR) to detect text in the subtitle. The authors in [3] presented InceptionV3 to generate visual subtitles. The study in [2] introduced a method combining Convolutional Neural Networks (CNNs) and Bidirectional Long Short-Term Memory (Bi-LSTM) Networks to capture the essential information of the sentence on subtitle documents from educational videos and text documents. The Spoken Subtitles initiative makes these subtitle-translated programs accessible to the visually impaired [13] by creating a service that automatically produces talking subtitles using synthetic speech.

142

H. T. Nguyen et al. Table 1. Data for testing

Name

Time

Size

Sample rate Channel

Audio1

2 m 37 s

26.5 MB 44100 Hz

Stereo

Audio2

2 m 53 s

29.2 MB 44100 Hz

Stereo

Audio3

1 m 26 s

19.5 MB 44100 Hz

Stereo

Audio from Google voice - female (en) 1 m 25 s

19.2 MB 44100 Hz

Stereo

Audio from Google voice - female (vi) 2 m 10 s

21.5 MB 44100 Hz

Stereo

Recorder file - female voice (en)

1 m 45 s

22.3 MB 44100 Hz

Stereo

Recorder file - female voice (vi)

1 m 17 s

16.8 MB 44100 Hz

Stereo

Recorder file - male voice (en)

1 m 29 s

20.6 MB 44100 Hz

Stereo

Audio from Google voice - male (en)

1 m 32 s

22.2 MB 44100 Hz

Stereo

Audio from Google voice - male (vi)

2 m 56 s

32.7 MB 44100 Hz

Stereo

Audio from Google voice - female (cn) 1 m 14 s

17.3 MB 44100 Hz

Stereo

Video1

1 m 41 s 146.4 MB 48000 Hz

Stereo

Video2

1 m 15 s 117.5 MB 48000 Hz

Stereo

Video2

2 m 59 s 238.4 MB 48000 Hz

Stereo

3

Data Description for Testing

In Table 1, We use the first three files: audio1, audio2, audio3, to test for noise reduction. The remaining files are used for speech-to-text, including noise reduction.

4 4.1

Proposed Approach Proposed Approach Model

Figure 1 includes five steps to perform noise reduction and automatic subtitle generation as follows: 1. Use FFmpeg to extract audio from the video file to create subtitles. 2. Put the extracted audio file in step 1 into the noise reduction process. 3. Convert audio file format from WAV to FLAC to meet the Speech to Text API service requirements. 4. Perform Speech-to-Text recognition based on the converted FLAC file in step 3 (which requires Internet access) to generate an SRT subtitle file in the form of text with timestamps. 5. Use FFmpeg to add subtitles (as an SRT file exported from step 4) to the original video. In this study, the Non-stationary Noise Reduction algorithm is applied to filter out the noise in the audio from [11]. In the following subsection, we will explain detail about noise reduction technique.

Evaluation on Noise Reduction in Subtitle Generator for Videos

143

Fig. 1. Proposed approach model

4.2

Noise Reduce Technique

As described in [11], Noisereduce is a noise reduction algorithm implemented in python to reduce noise in time-domain signals including speech, bioacoustics, and physiological signals. It relies on a “Spectral Gating” method, a form of Noise Gate. Noisereduce comprises two algorithms: Stationary Noise Reduction and Non-stationary Noise Reduction. The method calculated a spectrogram of a signal (and optionally a noise signal) and estimated a noise threshold (or gate) for each frequency band of that signal/noise. In addition, we can see that the threshold is used to compute a mask, which gates noise below the frequency-varying threshold.

Algorithm 1: Non-stationary Noise Reduction algorithm Begin Step 1: A spectrogram is calculated over the signal. Step 2: A time-smoothed version of the spectrogram is caculated with an IIR filter aplied forward and backward to each frequency channel. Step 3: A mask is computed based on that time-smoothed spectrogram. Step 4: The mask is smoothed with a filter over frequency and time. Step 5: The mask is appled to the spectrogram of the signal, and is inverted. End

144

5

H. T. Nguyen et al.

Evaluation and Testing

In this section, we evaluate two main functions of the proposed method: – Test the noise reduction processing for audio. – Test the process of recognizing audio converted to text Evaluate the results based on the output audio file that no longer contains noises such as traffic, music, animal sounds, etc., but only voices. - The audio files are represented through Maplotlib: after noise reduction processing, the audio segments must be reduced from the overlapping frequency ranges. - The audio files are shown in the Spek monitoring software: Remove the green, yellow, red spectrums, etc., the representations of noise. - Noise reduction processing speed.

6

Experimental Result

Fig. 2. Visualization of Audio1 before noise reduction

In Fig. 3 after processing the audio1 segment, a lot of noise, such as wind, animal, bell, running water, etc., have been removed from stereo (2 channels) to mono (1 channel). The green and yellow spectrums represent the noises that have been reduced quite a bit. Fast processing speed with 20 s.

Fig. 3. Visualization of Audio1 after noise reduction

Evaluation on Noise Reduction in Subtitle Generator for Videos

145

Fig. 4. Visualization of Audio2 before noise reduction

Fig. 5. Visualization of Audio2 after noise reduction

In Fig. 5 after processing the audio2 segment, much noise, such as wind, car, office, etc., have been removed from stereo (2 channels) to mono (1 channel). The green and yellow spectrums represent the noises that have been reduced quite a bit. Fast processing speed of 26 s.

Fig. 6. Visualization of Audio3 before noise reduction

In Fig. 7 after processing the audio3 segment, much noise, such as clapping hands, voice, etc., have been removed from stereo (2 channels) to mono (1 channel). The green and yellow spectrums represent the noises that have been reduced quite a bit. Fast processing speed with 12 s. In Figs. 2, 4, 6 visualization of Video1 was shown Spek1 and matplotlib [7]. The green and yellow spectrums are noise. 1

https://github.com/alexkay/spek, accessed on 10 March 2022.

146

H. T. Nguyen et al.

Fig. 7. Visualization of Audio3 after noise reduction Table 2. Result noise reduction on 3 audio files Name

Time processing Quality

Rating on spek Rating on matplotlib Result

Audio1 20 s

Noise reduction Good

Good

Pass

Audio2 26 s

Noise reduction Good

Good

Pass

Audio3 12 s

Noise reduction Good

Good

Pass

Table 2 shows very positive results, capable of eliminating noise reduction through noisereduce. In addition, all three files significantly reduce noise to improve the quality of the sound source. Table 3. Result of remaining files BEFORE noise reduction and Speech-to-Text

STT Audio & video

All word Detect word Correct word

1

Audio from Google voice - female (en) 101

94

93

2

Audio from Google voice - female (vi)

58

58

50

3

Recorder file - female voice (en)

81

69

40

4

Recorder file - female voice (vi)

52

48

45

5

Recorder file - male voice (en)

67

36

34

6

Audio from Google voice - male (en)

67

65

62

7

Audio from Google voice - male (vi)

52

56

50

8

Audio from Google voice - female (cn)

79

68

62

9

Video2

165

95

70

10

Video3

457

368

342

Table 3 shows that before noise reduction and Speech-to-Text conversion, it is found that the number of words detected in audio and video is not high because, in those files, there are different noises, affected partly through speech recognition. Table 4 shows that after noise reduction, it is found that the number of words recognized has increased. Furthermore, the rate has increased very positively by

Evaluation on Noise Reduction in Subtitle Generator for Videos

147

Table 4. Result of remaining files before AFTER reduction and Speech-to-Text STT Audio & video

All word Detect word Correct word Noise

1

Audio from Google voice - female (en) 101

94

94

2

Audio from Google voice - female (vi)

58

58

52

No No

3

Recorder file - female voice (en)

81

70

41

Wind, Human

4

Recorder file - female voice (vi)

52

52

48

Wind, Human

5

Recorder file - male voice (en)

67

58

55

Human, fan

6

Audio from Google voice - male (en)

67

66

66

No

7

Audio from Google voice - male (vi)

52

58

50

No

8

Audio from Google voice - female (cn)

79

78

72

9

Video2

165

102

100

Ringtone, clap

10

Video3

457

426

396

Music, vehicles, rain

No

Fig. 8. An illustration of application interface

Fig. 9. Result after processing a sample video

more than 70%, showing that the application of the noise reduction method increases the ability to recognize speech to apply Speech-to-Text API to convert speech to text to generate subtitles for video. From there, it is possible to create subtitles for video files to help viewers understand the content of the video quickly.

148

H. T. Nguyen et al.

Because the tool has a built-in noise reduction function, audio recognition into text is easier and more convenient. Therefore, the number of recognized words is relatively high, especially in the audio clips taken from Google’s voice, for the number of words to be recognized is close to the original sample’s number of words. In addition, because of using non-stationary noise reduction applications, the speech recognition rate has been raised relatively high. Moreover, with an accuracy of about 80%, the noise reduction works effectively, supporting the identification to occur smoothly. In general, all recognition results are at 70% or more and correct recognition is over 80% of the number of recognized words. The recognition based on audio clips is fast and entirely accurate because these audio clips contain only voices. Speak or prevent voice and noise but not much. The recognition of videos is more difficult because the videos have music or loud noises that overwhelm the voice, making identification difficult or impossible. In Fig. 8, an application was built-in PyQT52 . Manage input files support two languages for generating subtitles: Vietnamese English and in Fig. 9, an video output with subtitles has been added after processing, and files such as SRT subtitles, a text file containing the content of the subtitle, audio files extracted from the sample video are also created.

Fig. 10. Video before subtitle generator.

In Fig. 10, Video before using the proposed method. Because the sound quality in the sample video is low, there is much noise in the video like hand clapping, wind, vehicles, birds, etc. Therefore, the noise reduction in this video is still difficult as the API cannot accurately identify the sound in the video, so it is still guessing wrong or not. 2

https://pypi.org/project/PyQt5/.

Evaluation on Noise Reduction in Subtitle Generator for Videos

149

Fig. 11. video with generated subtitle.

In Fig. 10, video after using the proposed method, subtitles have been added to the video. Noises were reduced from the original video. It help API can detect voice easily and convert it to text to generate subtitle for this video.

7

Conclusion

In this study, we have evaluated some noise reduction to improve the quality of automatic subtitle generation through noise reduction processing in audio files, thereby using the Speech-to-Text API to create subtitles. Initial results are very positive in creating automatic subtitles to help viewers grasp the content of the video quickly. However, it is necessary to study other noise reduction techniques and improve Speech-to-Text recognition for better subtitle quality. In the future, we will improve processing time to create subtitles faster and can process videos longer than 5 min.

References 1. Aswin, V.B., et al.: NLP-driven ensemble-based automatic subtitle generation and semantic video summarization technique. In: Chiplunkar, N.N., Fukao, T. (eds.) Advances in Artificial Intelligence and Data Engineering. AISC, vol. 1133, pp. 3–13. Springer, Singapore (2021). https://doi.org/10.1007/978-981-15-3514-7 1 2. Chootong, C., Shih, T.K., Ochirbat, A., Sommool, W., Zhuang, Y.Y.: An attention enhanced sentence feature network for subtitle extraction and summarization. Expert Syst. Appl. 178, 114946 (2021). https://doi.org/10.1016/j.eswa.2021. 114946 3. Degadwala, S., Vyas, D., Biswas, H., Chakraborty, U., Saha, S.: Image captioning using inception v3 transfer learning model. In: 2021 6th International Conference on Communication and Electronics Systems (ICCES). IEEE (2021). https://doi. org/10.1109/icces51350.2021.9489111

150

H. T. Nguyen et al.

4. Domingo, I.V.R., Mamanta, M.N.G., Regpala, J.T.S.: FILENG: an automatic English subtitle generator from Filipino video clips using hidden Markov model. In: The 2021 9th International Conference on Computer and Communications Management. ACM (2021). https://doi.org/10.1145/3479162.3479172 5. Elshahaby, H., Rashwan, M.: An end to end system for subtitle text extraction from movie videos. J. Ambient Intell. Human. Comput. (2021). https://doi.org/ 10.1007/s12652-021-02951-1 6. Halpern, Y., et al.: Contextual prediction models for speech recognition. In: Proceedings of Interspeech 2016 (2016). http://www.isca-speech.org/archive/ Interspeech 2016/pdfs/1358.PDF 7. Hunter, J.D.: Matplotlib: a 2D graphics environment. Computi. Sci. Eng. 9(3), 90–95 (2007). https://doi.org/10.1109/MCSE.2007.55 8. Orero, P., Brescia-Zapata, M., Hughes, C.: Evaluating subtitle readability in media immersive environments. In: 9th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion. ACM, December 2020. https://doi.org/10.1145/3439231.3440602 9. Linhares Pontes, E., Gonz´ alez-Gallardo, C.-E., Torres-Moreno, J.-M., Huet, S.: Cross-lingual speech-to-text summarization. In: Choro´s, K., Kopel, M., Kukla, E., Siemi´ nski, A. (eds.) MISSI 2018. AISC, vol. 833, pp. 385–395. Springer, Cham (2019). https://doi.org/10.1007/978-3-319-98678-4 39 10. Roy, A., Phadikar, S.: Automatic segmentation of spoken word signals into letters based on amplitude variation for speech to text transcription. In: Mandal, J.K., Satapathy, S.C., Sanyal, M.K., Sarkar, P.P., Mukhopadhyay, A. (eds.) Information Systems Design and Intelligent Applications. AISC, vol. 340, pp. 621–628. Springer, New Delhi (2015). https://doi.org/10.1007/978-81-322-2247-7 63 11. Sainburg, T., Thielk, M., Gentner, T.Q.: Finding, visualizing, and quantifying latent structure across diverse animal vocal repertoires. PLoS Comput. Biol. 16(10), e1008228 (2020) 12. Seo, D., Gil, J.-M.: Speech-to-text-based life log system for smartphones. In: Park, D.S., Chao, H.C., Jeong, Y.S., Park, J. (eds.) Advances in Computer Science and Ubiquitous Computing. LNEE, vol. 373, pp. 637–642. Springer, Singapore (2015). https://doi.org/10.1007/978-981-10-0281-6 90 13. Verboom, M., Crombie, D., Dijk, E., Theunisz, M.: Spoken subtitles: making subtitled TV programmes accessible. In: Miesenberger, K., Klaus, J., Zagler, W. (eds.) ICCHP 2002. LNCS, vol. 2398, pp. 295–302. Springer, Heidelberg (2002). https:// doi.org/10.1007/3-540-45491-8 62 14. Victor, D.M., Eduardo, F.F., Biswas, R., Alegre, E., Fern´ andez-Robles, L.: Application of extractive text summarization algorithms to speech-to-text media. In: P´erez Garc´ıa, H., S´ anchez Gonz´ alez, L., Castej´ on Limas, M., Quinti´ an Pardo, H., Corchado Rodr´ıguez, E. (eds.) HAIS 2019. LNCS (LNAI), vol. 11734, pp. 540–550. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-29859-3 46 15. Yim, J.: Design of a subtitle generator. In: Advanced Science and Technology Letters. Science and Engineering Research Support soCiety, November 2015. https:// doi.org/10.14257/astl.2015.117.17

Hierarchical Output Model of CNN Learning Using Multi Label Datasets Jiha Kim, Agostinho Ant´ onio Jos´e, Jeena Kim, Yongho Kim, and Hyunhee Park(B) Department of Information and Communication Engineering, Myongji University, Seoul, South Korea {yaki5896,aajose,jnkim,yhkim98,hhpark}@mju.ac.kr

Abstract. In this paper, we propose a learning method for a deep learning model that becomes more complex as multi label datasets increase. The existing deep learning models are mainly models that classify each class in the final output layer. In other words, the results are output by ensuring the diversity of the learned models using the ensemble method. However, as the number of classification classes increase the number of network connections increase in the fully connected layer, resulting in frequent interference between nodes. As a result, it leads to overfitting that is impossible to learn. In this paper, to solve the problem above, a multi output model that learns the output layer independently and a hierarchical output model that learns hierarchically and outputs the results are proposed. Accuracy, f1-score, and ROC curve are used to compare the performance. Through this, the multi output model and the hierarchical output model show a performance improvement of about 8% and 10% compared to the existing single output model and about 3% and 5% compared to the ensemble model.

1

Introduction

With the recent increase in data collection, a dataset with one label per existing data uses one class to classify or predict data from nodes in the output layer. In addition, datasets with multi labels multiply the output layer by the number of classes present in each label to determine the number of output nodes. In other words, in the case of existing output models, N × M nodes are required in the output layer of the multi label dataset including N-class and M-class. In a single output model, the network becomes more complex as the number of output nodes increase. In this case, overfitting problem [1] that can cause learning problems arise. The solution is to limit the deep ensemble model [2], but overfitting problems arise because the number of nodes in the last output layer is the same as the single output model. In this paper, we propose several output models and hierarchical output models to solve this problem. Several output models have the same basic model and each consists of an independent model in the last output layer. Therefore, the N-class learning model and the M-class learning model have a common model c The Author(s), under exclusive license to Springer Nature Switzerland AG 2022  L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 151–160, 2022. https://doi.org/10.1007/978-3-031-08819-3_15

152

J. Kim et al.

and conduct separate learning. In the case of the hierarchical output model, after learning the gender classification model, it proceeds in the order of learning age according to the results. Learning with hierarchical models shows approximately 2% to 10% higher accuracy than conventional learning models, and up to 6% higher classification accuracy in the roc curve. Finally, Sect. 2 of this paper introduces the structures of existing learning models and proposes a hierarchical model. Section 3 shows the results of evaluating the performance of each model, and Sect. 4 we conclude with a discussion of the results that are presented in the current paper and an outlook on future work.

2

Proposed Training Method

In this section, the form and preprocessing phase of the dataset used the two existing models (single output model, ensemble model) and the proposed model (multi output model, hierarchical output model) are described. 2.1

Format of Dataset

In this paper, the dataset used for learning uses the utkface1 dataset of kaggle. The utkface dataset is a human face dataset and corresponds to a multi label dataset.

Fig. 1. Input image data and multi label information

Table 1. Information and description of each label

1

Labels

Output type

Range of data

Age

Linear

0 116 (integer type) Image age information

Description

Gender Classification [0–1] (2 classes)

Image gender information

Race

Image race information

Classification [0–4] (5 classes)

https://www.kaggle.com/jangedoo/utkface-new.

Hierarchical Output Model of CNN Learning Using Multi Label Datasets

153

In Fig. 1, labels are classified into age (red, linear data), gender (green, classification data), race (blue, classification data), and data index (gray, text data) of image data. The information of the class on each label is shown in Table 1. Table 2. Classification of age data Ages

Range of age

Age 0 [0–22] Age 1 [23–30] Age 2 [31–45] Age 3 [46–max]

The labels used as learning data use age and gender except race. Age information, which is linear data, is classified into four classes as shown in Table 2, that is, a total of 8 classes consisting of 4 age classes and 2 gender classes. The reason for classifying the age data as shown in Table 2 is to match the ratio of data used during learning phase. The image data use all 3-dimensional RGB values and each pixel is normalized to a value between 0 and 1. The purpose of the normalization is to centralize [3] the learning data. 2.2

Basic Models

Two models are used: a general single output model and an ensemble model. Figure 2(a) shows the configuration of a single output model. The final output layer consists of 8 nodes using the fully connected layer through global average pooling of EfficiencyNetB4 [4]. The structure of the ensemble model is shown in Fig. 2(b). The learning is done using a total of three different independent models (Vanilla CNN [5]). The ensemble model combines three models to implement a deep ensemble model. The output layer of the basic models compared in this paper consists of 8 nodes. This is made to take into account both age and gender classes. Therefore, if the learning model learns age, it is necessary to proceed with learning while considering gender also. For example, if the model has input label information of ‘25 year old’ and ‘male’, all labels should be considered. As a result, the network to which the node is connected to increases, and an overfitting problem occurs, and the model cannot learn.

154

J. Kim et al.

Fig. 2. Basic models structure (a) the structure of a single output model, and (b) the structure of a deep ensemble model

2.3

Multi Output Model

The configuration of the multiple output model can be verified as shown in Fig. 3. Based on EfficiencyNetB4, it is used as input for two models through global average pooling. The two models described in the paper are age classification model and a gender classification model, respectively. Interference between the models disappears because the models that classify age and gender learn as independent models. In other words, the learning age model only needs to learn age and does not need to consider the gender of the input data. Therefore, age labels composed of linear data can proceed with learning using linear activity functions as they are. 2.4

Hierarchical Output Model

Hierarchical output model are methods for sequentially learning independent models. Based on the results of the first learned model, the next model is learned. Figure 4 shows the process of learning the age after learning the gender in the learning environment of this work.

Hierarchical Output Model of CNN Learning Using Multi Label Datasets

155

Fig. 3. Structure of multi output model

Compared to other models, each model used for learning can be configured individually. This allows users to implement and use a dedicated model to learn each label. Unlike the ensemble method in Subsect. 2.2, each model outputs the results separately without aggregating the results of multiple models.

3

Simulation Results

This section deals with the analysis of the learning outcomes. The classification of the evaluation method is used to evaluate each learning result. To this end, the age linear prediction results in Subsects. 2.3 and 2.4 of Sect. 2 are divided into four categories within the range of Table 2. Therefore, it is classified into classes shown in Table 3. A total of three performance indicators are used such as, classification report, accuracy, ROC (receiver operating characteristic) curve [6] provided by Keras2 .

2

https://keras.io.

156

J. Kim et al.

Fig. 4. Structure of proposed hierarchical output model Table 3. Class type Class

Classify

Class 0 [0–22] Male Class 1 [0–22] Female Class 2 [23–30] Male Class 3 [23–30] Female Class 4 [31–45] Male Class 5 [31–45] Female Class 6 [46–max] Male Class 7 [46–max] Female

The learning proceeds in the server environment of the specifications shown in Table 4. Table 4. Environment of training Category Specification CPU

AMD Rayzen 7 3700x

GPU

NVIDIA GeForce GTX 1660 SUPER

RAM

32 GBytes

CUDA

11.0 ver

Hierarchical Output Model of CNN Learning Using Multi Label Datasets

3.1

157

Classification Report

The classification report function provided by Keras can output the overall performance indicators. The indicators used include precision, recall, and f1-score [7] and are output together with accuracy. Precision refers to the case where the true truth is true among those classified as true. Recall means the number of things that are actually true by the model. Precision and recall are calculated by Eqs. 1 and 2, respectively. (P recision) =

(Recall) =

TP TP + FP

(1)

TP TP + FN

(2)

Table 5. Simulation results Single output model

Ensemble model

Category Precision Recall F1-score Category Precision Recall F1-score Class 0

0.51

0.40

0.45

Class 0

0.58

0.51

0.54

Class 1

0.56

0.35

0.43

Class 1

0.60

0.54

0.57

Class 2

0.45

0.51

0.48

Class 2

0.51

0.61

0.55

Class 3

0.49

0.78

0.60

Class 3

0.62

0.63

0.62

Class 4

0.46

0.47

0.47

Class 4

0.51

0.58

0.54

Class 5

0.40

0.20

0.26

Class 5

0.40

0.51

0.45

Class 6

0.63

0.75

0.68

Class 6

0.76

0.65

0.70

Class 7

0.60

0.42

0.50

Class 7

0.69

0.49

0.57

Average

0.51

0.49

0.48

Average

0.58

0.56

0.57

Weighted 0.51

0.52

0.50

Weighted 0.59

0.57

0.58

Accuracy 0.52

Accuracy 0.57

Multi output model

Hierarchical output model

Category Precision Recall F1-score Category Precision Recall F1-score Class 0

0.75

0.52

0.62

Class 0

0.95

0.64

0.76

Class 1

0.59

0.69

0.63

Class 1

0.81

0.60

0.69

Class 2

0.53

0.41

0.47

Class 2

0.53

0.50

0.51

Class 3

0.55

0.64

0.59

Class 3

0.58

0.68

0.62

Class 4

0.47

0.67

0.55

Class 4

0.44

0.67

0.53

Class 5

0.45

0.43

0.44

Class 5

0.43

0.51

0.47

Class 6

0.80

0.73

0.76

Class 6

0.85

0.66

0.74

Class 7

0.82

0.57

0.67

Class 7

0.87

0.68

0.76

Average

0.62

0.58

0.59

Average

0.68

0.62

0.64

Weighted 0.62

0.60

0.60

Weighted 0.67

0.62

0.63

Accuracy 0.60

Accuracy 0.62

158

J. Kim et al.

F1-score means the average of precision and recall, and it can be seen how appropriate the two performances are. This is calculated by Eq. 3. (F 1 − score) = 2 ×

1 1 P recision

1 + Recall P ecision × Recall =2× P recision + Recall

(3)

Table 5 shows the performance of the test set. According to the overall learning results, the hierarchical output model shows 2–10% higher performance than the other models. Multiple output models and hierarchical output models are designed with a relatively simpler network than conventional learning method s. This is because one model per label proceeds with the learning. As the network becomes simpler, each node interference decreases and the learning performance improves. The accuracy is improved by about 2% compared to the multi output model of the same output type, but for the f1-score, the hierarchical output model shows a 5% higher performance improvement. The reason is that in the case of multi output model, the learning model per label is independent, but the underlying model used is the same. As a result, it shows low performance of 2% to 5%. 3.2

ROC Curve

Figure 5 shows the result of the ROC curve region for each class for the four models. The ROC curve is used to show the performance of the classification model. The method of showing this performance is represented by the area under the graph specified by Area Under the ROC (AUC) curve. Figure 5(a) and (b) show the results of learning with basic models. Overall, in the ensemble model of Fig. 5(b), most classes of AUC have a range of 70–80%. Since this is the result of output using the harmony of learning results of various models, it can be seen that the AUCs of all classes are similar. On the other hand, the AUCs of classes 3 and 6 in Fig. 5(a) are 82% and 83%, which are particularly high. This shows that the single output model shows that networks classifying classes 3 and 6 can affect other nodes, making learning of other classes difficult. Figure 5(c) and 5(d) show that most classes are higher than the AUC of the basic models. In addition, the average AUC for all classes represents the highest performance from the hierarchical output model, which is 78%. This can be attributed to the use of independent learning models for each label to reduce network complexity and minimize the impact on other nodes, unlike the basic model learning methods.

Hierarchical Output Model of CNN Learning Using Multi Label Datasets

159

Fig. 5. ROC curve results for each learning model

4

Conclusion

From the result, it seems that the network of the model should not be complicated to effectively learn the multi label dataset. To this end, this paper proposed a hierarchical output model in which each label is learned hierarchically. This model shows a performance improvement of 2–10% in terms of accumulation compared to conventional learning methods. In addition, the f1-score is about 5% higher than several output models of the same output type. However, since the learning method of the hierarchical output model is not elaborately made, the two models are sequentially learned, and the results are output. To compensate for this, a more sophisticated hierarchical learning architecture will be used in future works. The goal is to further improve learning outcomes for different datasets, including multi labels.

160

J. Kim et al.

Acknowledgements. This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2022R1A2C2005705, AI-MAC Protocol on Distributed Machine Learning for Intelligent Flying Base Station).

References 1. Ying, X.: An overview of overfitting and its solutions. J. Phys. Conf. Ser. 1168, 022022 (2019) 2. An, N., Ding, H., Yang, J., Au, R., Ang, T.F.: Deep ensemble learning for Alzheimer’s disease classification. J. Biomed. Inf. (2020). https://doi.org/10.1016/ j.jbi.2020.103411 3. Sola, J., Sevilla, J.: Importance of input data normalization for the application of neural networks to complex industrial problems. IEEE. Trans. Nuclear Sci. 44, 1464–1468 (1997) 4. Tan, M., Le, Q.: EfficientNet: rethinking model scaling for convolutional neural networks. PMLR 97, 6105–6114 (2019) 5. Albawi, S., Mohammed, T.A., Al-Zawi, S.: Understanding of a convolutional neural network. In: International Conference on Engineering and Technology (2017). https://doi.org/10.1109/ICEngTechnol.2017.8308186 6. Hand, D.J., Till, R.J.: A simple generalisation of the area under the ROC curve for multiple class classification problems. Mach. Learn. (2001). https://doi.org/10. 1023/A:1010920819831 7. Goutte, C., Gaussier, E.: A probabilistic interpretation of precision, recall and F score, with implication for evaluation. In: Losada, D.E., Fern´ andez-Luna, J.M. (eds.) ECIR 2005. LNCS, vol. 3408, pp. 345–359. Springer, Heidelberg (2005). https://doi. org/10.1007/978-3-540-31865-1 25

Efficient Privacy-Preserving Authentication and Group Key Agreement Scheme in Fog-Enabled VANET Cong Zhao1 , Nan Guo2(B) , and Tianhan Gao1 1

2

Software College, Northeastern University, Shenyang, China [email protected] Computer Science and Engineering College, Northeastern University, Shenyang, China [email protected]

Abstract. In order to support the construction of intelligent transportation system (ITS), Vehicular Ad-hoc Network (VANET) comes into being and is attracting the attention from both the industry and research communities. VANET is a special mobile ad-hoc network that allows vehicles to communicate with Roadside Units (RSUs) or other vehicles, which can alleviate traffic congestion and avoid accidents. However, as for the open environment of VANET, vehicles are more vulnerable to attacks. Meanwhile, VANET is a dynamic and time-critical network, therefore minimizing latency becomes the core concern. Using fog nodes is an attractive method to solve the problem since it can reduce communication delay and increase throughput. To maintain the balance between privacy and security in fog-enabled VANET, this paper proposes an efficient privacypreserving authentication and group key management scheme. Fog nodes cooperate with RSUs to establish groups and generate group keys. The proposed scheme prevents vehicles’ sensitive information from being disclosed. Moreover, the scheme guarantees the communication always be anonymous. When a vehicle is found to have misbehaved, the real identity of the vehicle can be retrieved. The security and performance analysis proves that the proposed scheme is privacy-preserving, secure and efficient.

1

Introduction

The number of vehicles has increased dramatically recently. Therefore, many issues such as traffic jams and collisions are prominent. In order to support the construction of intelligent transportation system (ITS), Vehicular Ad-hoc Network (VANET) comes into being and quickly attracts the attention from both the industry and research communities, which can mitigate traffic congestion and avoid accidents [1]. c The Author(s), under exclusive license to Springer Nature Switzerland AG 2022  L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 161–171, 2022. https://doi.org/10.1007/978-3-031-08819-3_16

162

C. Zhao et al.

VANET is a special mobile ad-hoc network that allows vehicles in the network can communicate directly and conveniently. Vehicles can communicate with others through Vehicle-to-Vehicle (V2V) and with Roadside Units (RSUs) through Vehicle-to-Infrastructure (V2I) [2]. Each vehicle is equipped with a tamperresistant On-Board Unit (OBU). There are various onboard sensors equipped on the OBU to collect surrounding data for vehicles. Vehicles broadcast beacons every 100 to 300 ms based on the Dedicated Short Range Communication protocol (DSRC) during driving [3]. The beacon always contains traffic-related and surrounding information, such as location, speed, direction, and road conditions. Using this kind of message can realize traffic congestion mitigation and accidents avoidance, thus improving the safety of drivers and traffic efficiency. As for the open environment of VANET, vehicles are more vulnerable to being attacked. Attackers can process the transmitted safety related information, such as intercepting, tampering, replaying, deleting, and other ways [4]. These attacks may cause immeasurable damage to the whole network, even threatening the drivers’ and passengers’ property and lives. Therefore, protecting the security and privacy of vehicles becomes the top priority in VANET [5–8]. Both V2V and V2I communications must be anonymous. Moreover, the real identity of the vehicle must be able to be retrieved. Meanwhile, VANET is a dynamic, time-critical network, which is different from other open and insecure wireless networks, making the traditional authentication scheme not suitable for VANET [9]. With the increase of connection vehicles in VANET, a large amount of data will be generated at the edge of the network. In this case, minimizing latency becomes the core concern. Using fog nodes is an attractive method because it reduces communication delay and increases throughput [7]. Therefore, this paper deploys fog nodes in VANET. To maintain the balance between privacy and security in fog-enabled VANET, this paper proposes an efficient privacy-preserving authentication and group key management scheme. The proposed scheme prevents vehicles’ sensitive information from being disclosed. Fog nodes cooperate with RSUs to establish groups and generate group keys for group members. The remainder of the paper is organized as follows. Section 2 introduces the Preliminaries. The system model and the construction of the proposed scheme are presented in Sect. 3. In Sect. 4, security and performance analysis is elaborated. Finally, the conclusion of the scheme is provided in Sect. 5.

2 2.1

Preliminaries Chinese Remainder Theorem

The Chinese Remainder Theorem (CRT) means that if one knows the remainder of an integer n divided by several integers, the one can uniquely determine the remainder of n divided by the product of these integers, under the condition that the divisors are pairwise coprime [10].

Authentication Scheme in Fog-Enabled VANET

163

Assuming that the integers a1 , a2 , ..., an are pairwise prime, then for any integer b1 , b2 , ..., bn , states that the pair of congruences, x ≡ b1 mod a1

(1)

x ≡ b2 mod a2

(2)

... (3) x ≡ bn mod an n has a unique solution mod ∂ = a1 a2 ...an = i=1 ai . The value of the unique solution can be computed as : x = b1 + b2 + ... + bn mod ∂ =

n 

bi βi γi mod ∂

(4)

i=1

where βi = 2.2

∂ ai

and βi γi = 1 mod ai .

CC Signature

Initialization. Suppose G1 is an additive cyclic group of order p generated by P . GT is a multiplicative cyclic group with the same order p. Define a bilinear mapping e : G1 ×G1 → GT and two secure hash functions H1 : {0, 1}∗ → G1 and H2 : {0, 1}∗ × G1 → Zp∗ . CA randomly selects a master private key s ∈ Zp∗ and computes Ppub = sP . The system parameters are {G1 , GT , p, e, P, Ppub , H1 , H2 }. Key Extraction. Given the identity ID of a user, CA computes the public key and private key of the user Q = H1 (ID) and S = sQ. Signature. Given a message M , the signer randomly selects r ∈ Zp∗ and computes V = rQ, h = H2 (M, V ) and W = (r + h)S. The signature of the message M is σ = (V, W ). Verification. After receiving the message M and the corresponding signature σ, the verifier computes h = H2 (M, V ) and verifies that equation e(P, W ) = e(Ppub , V +hQ) holds. If the above formula is true, the signature σ is a legitimate signature. Otherwise, the signature σ is not valid.

3

The Proposed Scheme

This section presents the system model and elaborates different phases of the proposed scheme. To satisfy the security and privacy requirements of VANET, this paper proposes an efficient privacy-preserving authentication and group key management scheme. There are mainly four phases including system initialization, group key generation, secure communication, and group key updation.

164

C. Zhao et al.

3.1

System Model

According to Fig. 1, the system model includes Certificate Authority (CA), Fog Nodes (FNs), Roadside Units (RSUs), and On-Board Units (OBUs). Certificate Authority (CA): CA is considered as a trusted authority, which is responsible for issuing certificates, public and private keys, and system parameters to FNs, RSUs and OBUs. Besides, CA is also responsible for the security management, revoking nodes when vehicles broadcast false messages or perform malicious behaviors. CA has high computation capacity and sufficient storage capacity. Fog Node (FN): Each fog node, as a group manager, manages all vehicles within the coverage area of multiple RSUs. Fog nodes only process messages related to group management from RSUs. The computation capacity and storage capacity of fog nodes are higher than RSUs but lower than CA. Roadside Unit (RSU): RSUs are usually deployed along roads or in hot spots, such as intersections, schools or shopping malls. RSUs are responsible for forwarding messages to OBUs or FNs to extend the communication range and collecting beacons for security applications, such as malicious vehicle reports. On-Board Unit (OBU): OBU is installed on each vehicle to communicate with adjacent vehicles and RSU through wireless media. OBUs are composed of storage devices, network devices and sensors. Global Positioning System (GPS) and other sensors collect information and send it to OBUs. 3.2

System Initialization

1) System Setup: CA selects a large prime number q, where q is used to define the multiplicative group Zq∗ . CA defines an additive cyclic group G1 of order

Fig. 1. System model.

Authentication Scheme in Fog-Enabled VANET

165

p generated by P and a multiplicative cyclic group GT with the same order p. Meanwhile, CA defines a bilinear mapping e : G1 ×G1 → GT and four secure hash functions H1 : {0, 1}∗ ×Zp∗ → G1 , H2 : {0, 1}∗ ×{0, 1}∗ ×G1 ×G1 → Zp∗ , H3 : G1 → {0, 1}∗ and H4 : {0, 1}∗ × {0, 1}∗ × G1 → Zp∗ . CA randomly chooses a master private key s ∈ Zp∗ and computes Ppub = sP . The system parameters are {G1 , GT , p, q, e, P, Ppub , H1 , H2 , H3 , H4 }. 2) Vehicle Registration: Each vehicle sends its real identity ID to CA. CA chooses a random number r1 ∈ Zp∗ and calculates the private key Sv = sQv , where Qv = H1 (ID||r1 ) is the public key of the vehicle. For generating pseudonyms for each vehicle, CA computes P ID1 = r1 P , P ID2 = ID ⊕ H3 (r1 Ppub ). Then CA chooses a random number r2 ∈ Zp∗ and computes V = r2 Ppub , h = H2 (P ID1 ||P ID2 ||V ||T ) and W = (r2 + h)sPpub , where T is the valid period. CA sends the public and private key (Qv , Sv ), pseudonym (P ID1 , P ID2 ) and certificate (W, V, T ) through secure channels to vehicles. After receiving the certificate, the vehicle verifies that equation e(P, W ) = e(Ppub , V + hPpub ) holds. If the verification is success, the vehicle stores the certificate. Otherwise, discards it. 3.3

Group Key Generation

When the vehicle enters the communication range of a RSU, the vehicle broadcasts its pseudonym, certificate, the group joining request and the corresponding signature to the nearest RSU: 1) Given a group joining request Req, the vehicle chooses a random number r ∈ Zp∗ . Then the vehicle computes V  = r Qv , h = H4 (Req||V  ||ts) and W  = (r + h )Sv , where ts is the current timestamp. Afterwards, the vehicle sends the packet {W, V, T, Qv , W  , V  , P ID1 , P ID2 , ts, Req} to the nearest RSU. 2) The RSU computes h1 = H2 (P ID1 ||P ID2 ||V ||T ) and verifies that equation e(P, W ) = e(Ppub , V + h1 Ppub ) holds. If the verification is success, the RSU sends the packet {Qv , W  , V  , ts, Req} to the nearest FN. 3) Assume that the FN receives m signatures of multiple vehicles from different RSUs. The FN first computes hi  =H4 (Reqi ||Vi  ||tsi )(i  = 1, 2, ..., m). Then m m the FN verifies that equation e(P, i=1 Wi  ) = e(Ppub , i=1 (Vi  + hi  Qv i ) holds. If the verification is successful, the FN executes Step 4–5 to generate a new group key for requested vehicles. 4) FN chooses random numbers ski ∈ Zq∗ for requested vehicles. Then FN n computes ∂ = sk1 sk2 ...skn = i=1 ski and xi = sk∂ i , where n is the number such that of vehicles in the group managed by FN. Next, FN computes yi N xi ×yi ≡ 1 mod ski . Finally, FN computes vari = xi ×yi and μ = i=1 vari . 5) Assuming that every interval, FN chooses a random number gk ∈ Zq∗ as the new group key and computes GK = gk × μ. FN signs GK and the valid period of this group key TG using its private key and broadcasts {GK, SIGF N (GK||TG )} to all RSUs in its domain. Meanwhile, FN encrypts ski with the public key Qv i of each vehicle. Finally, the FN sends Encvi (ski ) to vehicles through RSUs.

166

C. Zhao et al.

6) Vehicle decrypts Encv (sk) with the private key Sv and stores the sk. Then vehicle verifies the signature SIGF N (GK||TG ) and store GK, TG . The vehicle can obtain the new group key by computing gk = GK mod sk. Finally, vehicles in the same group can communicate with each other using the group key. 3.4

Secure Communication

After the vehicle successfully joins the group, the message sender can communicate with other entities by the following steps: Given a message m, the sender encrypts m with the group key Encgk (m||ts), where ts is the current timestamp. Then the sender sends the packet {W, V, T, P ID1 , P ID2 , ts, m, Encgk (m||ts)} to the receiver. Assume that the receiver receives m packets from multiple senders, the = 1, 2, ..., m) and verreceiver first computes h 2 i = H2 (P ID1 i ||P ID 2 i ||Vi ||Ti )(i  m m m ifies that equation e(P, i=1 Wi ) = e(Ppub , i=1 Vi + ( i=1 h2i )Ppub ). If the verification is successful, the receiver accepts the message. Otherwise, refuses it. 3.5

Group Key Update

When new vehicles join or leave the group, the group key needs to be updated. 1.a) When a vehicle Vjoin joins the group, FN first executes Step 3 in Sect. 3.3 to authenticate the vehicle. Then FN randomly chooses skjoin and computes  ∂  = ∂skjoin and xjoin = sk∂join . Afterwards, FN computes yjoin such that xjoin × yjoin ≡ 1 mod skjoin and computes varjoin = xjoin × yjoin . Finally, FN chooses a new group key gk  and computes μ = μ + varjoin , GK  = μ × gk  . 1.b) When a vehicle Vlea leaves the group, FN computes μ = μ − varlea and GK  = μ × gk  . 2) FN broadcasts the new parameters and signatures to all vehicles in the group. Vehicles use these parameters to obtain the new group key.

4

Security and Performance Analysis

In this section, the security and privacy analysis and performance analysis are introduced in details. 4.1

Security and Privacy Analysis

Minimum Disclosure. In the group key generation phase, the vehicle sends its pseudonym (P ID1 , P ID2 ), public key Qv , certificate (W, V, T ), the group joining request Req and the corresponding signature (W  , V  , ts) to the nearest RSU. Pseudonym (P ID1 , P ID2 ) and public key Qv are randomly generated based on random number r1 and the unique identity ID. Certificate (W, V, T ) and

Authentication Scheme in Fog-Enabled VANET

167

the corresponding signature (W  , V  , ts) are generated based on the pseudonym (P ID1 , P ID2 ) and the group joining request Req, respectively. Meanwhile, certificate (W, V, T ) and the corresponding signature (W  , V  , ts) are also generated based on random numbers and timestamps. The packet does not disclose any additional personal sensitive information of vehicles. Therefore, the proposed scheme satisfies minimum disclosure. Message Integrity. In the proposed scheme, message integrity is guaranteed by the identity-based CC signature. The certificate (W, V, T ) issued by CA can be verified by the message receiver. Meanwhile, all messages and signatures (W  , V  , ts, m) sent by vehicles needs to be verified. As shown in Sect. 3.3, the packet (W, V, Qv , W  , V  , P ID1 , P ID2 , ts, T, Req) needs to be verified by equations e(P, W ) = e(Ppub , V + hPpub ) and e(P, W  ) = e(Ppub , (V  + h Qv )). If validation fails, the packet is discarded. Therefore, the scheme satisfies message integrity Unlinkability. In the group key generation phase, the vehicle generates signature (W  , V  , ts) based on random numbers and the current timestamp. The pseudonym (P ID1 , P ID2 ), public key Qv and certificate (W, V, T ) issued by CA are also generated based on random numbers and timestamps. These parameters used in each signature process are different, hence, the packets sent by vehicles are also different, that is, the proposed scheme satisfies unlinkability. Anonymity. The pseudonym (P ID1 , P ID2 ), public key Qv , certificate (W, V, T ) and the corresponding signature (W  , V  , ts) do not contain additional information, nobody is able to access the vehicle’s real identity through these parameters. Attackers must solve the CDH cryptographic problem and break through one-way hash functions to get the true identity ID of vehicles. Therefore, our proposed scheme provides identity privacy protection and satisfies the requirement of anonymity. Backward Security and Forward Security. Assuming that every interval, FN will automatically update the group key, and the new group key is not related to the previous group key. After obtaining the new group key, the newly joined vehicle can access messages in the group, but cannot use the new group key to obtain the previous information. Once the vehicle leaves the group, the corresponding parameter varlea will be removed from the variable μ. The vehicles who leave the group cannot obtain the new group key, therefore, they cannot access the latest messages in the group. Therefore, the proposed scheme guarantees both backward security and forward security. Accountability. The vehicle uses pseudonym (P ID1 , P ID2 ) and certificate (W, V, T ) to communicate with other vehicles. Only the CA can retrieve identities of vehicles through the pseudonym by a XOR: ID = H3 (sP ID1 ) ⊕ P ID2 .

168

C. Zhao et al.

Consequently, vehicles cannot deny messages sent by themselves, therefore our scheme satisfies the accountability. 4.2

Performance Analysis

Due to the limited computation capacity of vehicles, the proposed scheme is evaluated from two aspects: computation overhead and communication overhead. We demonstrate the performance analysis of the proposed scheme in detail against EAAP [11] and CPAS [12]. 4.2.1

Computation Overhead

In VANET, real-time is very important, therefore we need to consider the computation overhead in communication. The notation of execution time of basic cryptographic operations used in the proposed scheme and compared schemes are shown in Table 1. Note that the paper only focus on time-consuming operations, e.g., Tpm , Tbp , while the lightweight operations, such as Tpa , Te /Td and Th are ignored [13]. Table 1. Notation of execution time. Notation Descriptions Tbp

A bilinear pairing operation

Tpm

A point multiplication operation related to bilinear pairing

Tpa

A point addition operation related to bilinear pairing

Th

A secure hash function operation

Te /Td

A symmetric encryption/decryption operation

In the phase of group key generation, given the group joining request, the vehicle executes two point multiplication operations to generate a signature. The computation overhead of the vehicle joining the group is 2Tpm . Then the vehicle sends the packet to the nearest RSU. The RSU executes one point multiplication operation and two bilinear pairing operations to verify the received certificate. The computation overhead of the RSU is Tpm +2Tbp . If the verification is successful, the RSU forwards received signatures to the FN. Assume the FN received m signatures, FN requires m point multiplication operations and two bilinear pairing operations to verify received signatures. The computation overhead of the FN verifies m signatures is mTpm + 2Tbp . In the phase of secure communication, the message sender in the group only requires one symmetric encryption operation to communicate, and the encryption operation is negligible. Assume the receiver received m packets, the receiver requires m point multiplication operations and two bilinear pairing operations to verify received packets. The computation overheads is mTpm + 2Tbp .

Authentication Scheme in Fog-Enabled VANET

169

EAAP [11]: Given system parameters and key pairs (Yk , rk ), the sender requires 6 point multiplication operations to generates its certificate and signature. When getting messages, the verifier requires 5 point multiplication operations and two bilinear pairing operations to verify one signature and certificate or requires (m + 4) point multiplication operations and (m + 1) bilinear pairing operations to verify m signatures and certificates. CPAS [12]: The sender requires three point multiplication operations to sign a message M with private key SKi and system parameters. The verifier requires two point multiplication operations and three bilinear pairing operations to validate a single message or requires (m + 1) point multiplication operations and three bilinear pairing operations to validate multiple messages simultaneously. The comparison of computation overhead is shown in Table 2. Table 2. Comparison of computation overhead. V2I

V2V

Batch verification

Inspection

EAAP [11] 11Tpm + 2Tbp 11Tpm + 2Tbp (m + 4)Tpm + (m + 1)Tbp 2Tpm CPAS [12] 5Tpm + 3Tbp

5Tpm + 3Tbp

(m + 1)Tpm + 3Tbp

XOR

Ours

Tpm + 2Tbp

mTpm + 2Tbp

XOR

4.2.2

4Tpm + 4Tbp

Communication Overhead

In VANET, the wireless channel is one of the main bottlenecks of data transmission. Therefore, this paper focuses on the communication overhead brought by vehicle communication. The communication overhead includes some authentication-related information such as pseudonyms, signatures, certificates, and timestamps (messages are ignored). Assume that the size of p is 64 bytes, which means the size of the element in G1 is 128 bytes. We also assume the size of the hash function, timestamp and the size of q is 20, 4, 20 bytes, respectively [14]. The packet sent by vehicle to join the group is (W, V, Qv , W  , V  , P ID1 , P ID2 , ts, T, Req), where W, V, Qv , W  , V  , P ID1 ∈ G1 P ID2 ∈ Zq∗ and T, ts are timestamps, therefore the communication overhead is 6|G1 | + |Zq∗ | + 2|T | = 796bytes. The packet sent by message sender in the group to communicate is (W, V, T, P ID1 , P ID2 , ts, m, Encgk (m||ts)), where W, V, P ID1 ∈ G1 P ID2 ∈ Zq∗ and T, ts are timestamps, hence the communication overhead is 3|G1 | + |Zq∗ | + 2|T | = 412bytes. As described in Sect. 4.2.1, the computation overhead of EAAP and CPAS are COEAAP = 7|G1 |+4|Zq∗ | = 976bytes and COCP AS = 5|G1 |+|T | = 644bytes, respectively. The comparison of communication overhead is shown in Table 3.

170

C. Zhao et al. Table 3. Comparison of communication overhead. V2I (bytes) V2V (bytes)

5

EAAP [11] 976

976

CPAS [12] 644

644

Ours

412

796

Conclusions

This paper proposes an efficient privacy-preserving authentication and group key management scheme for secure communication in VANET. To minimize the communication latency, this paper deploys fog nodes to cooperate with RSUs to establish groups for vehicles. The proposed scheme can prevent vehicles’ sensitive information from being disclosed to others. Batch verification is also supported in the scheme. Besides, the adoption of the Chinese Remainder Theorem has been proven to improve transmission efficiency. The security and privacy analysis proves that the proposed scheme can satisfy security and privacy requirements, such as minimum disclosure, integrity, backward security and forward security, and so on. Moreover, the performance analysis shows that the proposed scheme is efficient. In the future, we will replace the CC signature with a more efficient one and refine and simulate the proposed scheme. Acknowledgements. This work was supported by National Natural Science Foundation of China under Grant Number 52130403 and Fundamental Research Funds for the Central Universities under Grant Number N2017003.

References 1. Muniyandi, R.C., Qamar, F., Jasim, A.N.: Genetic optimized location aided routing protocol for VANET based on rectangular estimation of position. Appl. Sci. 10(17), 5759 (2020) 2. Yang, J., Deng, J., Xiang, T., Tang, B.: A Chebyshev polynomial-based conditional privacy-preserving authentication and group-key agreement scheme for VANET, May 2021 3. Al-shareeda, M., Anbar, M., Hasbullah, I.: Manickam, S.: Survey of authentication and privacy schemes in vehicular ad hoc networks. IEEE Sens. J. 1 (2020) 4. Cui, J., Zhang, X., Zhong, H., Zhang, J., Liu, L.: Extensible conditional privacy protection authentication scheme for secure vehicular networks in a multi-cloud environment. IEEE Trans. Inf. Forensics Secur. 15, 1654–1667 (2020) 5. Mejri, M.,N. Ben-Othman, J., Hamdi, M.: Survey on VANET security challenges and possible cryptographic solutions. Veh. Commun. 1(2), 53–66 (2014). https:// www.sciencedirect.com/science/article/pii/S2214209614000187 6. Altaf, F., Maity, S.: Plhas: privacy-preserving localized hybrid authentication scheme for large scale vehicular ad hoc networks. Veh. Commun. 30, 100347 (2021). https://www.sciencedirect.com/science/article/pii/S2214209621000164

Authentication Scheme in Fog-Enabled VANET

171

7. Soleymani, S.A., Goudarzi, S., Anisi, M.H., Zareei, M., Abdullah, A.H., Kama, N.: A security and privacy scheme based on node and message authentication and trust in fog-enabled VANET. Veh. Commun. 29, 100335 (2021). https://www. sciencedirect.com/science/article/pii/S2214209621000048 8. Al-Heety, O.S., Zakaria, Z., Ismail, M., Shakir, M.M., Alani, S., Alsariera, H.: A comprehensive survey: benefits, services, recent works, challenges, security, and use cases for SDN-VANET. IEEE Access 8, 91 028–91 047 (2020) 9. Wang, Y., Zhang, W., Wang, X., Khan, M.K., Fan, P.: Efficient privacy-preserving authentication scheme with fine-grained error location for cloud-based VANET. IEEE Trans. Veh. Technol. 70(10), 10 436–10 449 (2021) 10. Vijayakumar, P., Bose, S., Kannan, A.: Chinese remainder theorem based centralised group key management for secure multicast communication. Inf. Secur. (2014) 11. Azees, M., Vijayakumar, P., Deboarh, L.J.: EAAP: efficient anonymous authentication with conditional privacy-preserving scheme for vehicular ad hoc networks. IEEE Trans. Intell. Transp. Syst. 18, 2467–2476 (2017) 12. Shim, K.-A.: CPAS: an efficient conditional privacy-preserving authentication scheme for vehicular sensor networks. IEEE Trans. Veh. Technol. 61, 1874–1883 (2012) 13. Kong, Q., Lu, R., Ma, M., Bao, H.: A privacy-preserving and verifiable querying scheme in vehicular fog data dissemination. IEEE Trans. Veh. Technol. 68(2), 1877–1887 (2019) 14. Ali, I., Li, F.: An efficient conditional privacy-preserving authentication scheme for vehicle-to-infrastructure communication in VANETs. Veh. Commun. 22, 100228 (2020). https://www.sciencedirect.com/science/article/pii/S221420961930275X 15. Liu, Z.-C., Xiong, L., Peng, T., Peng, D.-Y., Liang, H.-B.: A realistic distributed conditional privacy-preserving authentication scheme for vehicular ad hoc networks. IEEE Access 6, 26 307–26 317 (2018)

3D Reconstruction Based on the Depth Image: A Review Qingwei Mi and Tianhan Gao(B) Software College, Northeastern University, Shenyang, China [email protected], [email protected]

Abstract. Three-dimensional (3D) reconstruction is an important field of computer vision. Though Image-based 3D reconstruction is more widely used due to its low environmental requirements, current research on 3D reconstruction based on the depth image is still very limited and many aspects need improvements. This paper reviews the basic process of 3D reconstruction technology based on the depth image and introduces relevant technologies in detail. The core algorithms and methods are analyzed by comparing advantages and disadvantages and representative research works in recent years are concluded. Furthermore, future research prospects of 3D reconstruction based on the depth image are proposed by analysis of hot spots, difficulties, and possible development trends in technical research to provide support for relevant researchers.

1 Introduction Three-dimensional (3D) reconstruction is the process of capturing the shape and appearance of real objects, which aims to construct a mathematical model suitable for representation and processing of computers. In computer environments, 3D reconstruction is the basis for processing, manipulating and analyzing model properties, as well as a key technology to build virtual scenes to display the real world [1, 2]. Nowadays, 3D reconstruction has been widely applied to multitudinous fields, such as computer vision, virtual reality, robot navigation, reverse engineering, etc. It also plays an important role in cultural relics restoration, architecture design, medical imaging processing, and film and animation production. According to the way of acquiring 3D information, 3D reconstruction can be classified into image-based, software-based, and hardware-based reconstruction. With the maturity of depth cameras, depth images are increasingly used for 3D reconstruction. The paper mainly studies the 3D reconstruction technologies based on the depth image. The main idea of which is to use the depth camera to scan the object to obtain its depth image, and use the depth data to realize the model reconstruction. Compared with the reconstruction based on two-dimensional (2D) image information, the method based on the depth image reduces the investment of manpower and material resources greatly, saves much time and improves efficiency in technology development and application. The remainder of the paper will discuss the key technical issues of 3D reconstruction from the analysis of the existing technologies and future development directions which is organized as follows. In Sect. 2, the basic process of 3D reconstruction based on the © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 172–183, 2022. https://doi.org/10.1007/978-3-031-08819-3_17

3D Reconstruction Based on the Depth Image: A Review

173

depth image and related theories are introduced. Section 3 elaborates the representative relevant methods and algorithms in academia and industry. Finally, the paper is concluded to propose future research directions in Sect. 4.

2 Related Technologies The process of 3D reconstruction based on the depth image includes five key technologies, which are depth image preprocessing, point cloud computing, registration, fusion, and model surface generation. For the existing depth image, the target 3D model is obtained to complete the 3D reconstruction by the basic process shown in Fig. 1.

Fig. 1. Basic process of 3D reconstruction based on the depth image.

2.1 Preprocessing Due to external factors such as illumination during shooting, the resolution of the shooting equipment, the positional relationship between the object and the device, and the surface condition of the object to be tested, the depth image given as input cannot usually be used for the 3D reconstruction process directly. To make the image suitable for reconstruction, the primary scheme is to preprocess, which includes eliminating noise and invalid pixels and fixing missing pixels to enhance the image usability. In order to preserve the existing information of the depth image as possible in the process of denoising, the obtained data should be filtered properly. Mean filtering, Gaussian filtering, median filtering, etc. are filtering algorithms which used commonly at present. Moreover, the missing depth pixels of the depth image can be filled effectively by using Fast Marching Method (FMM) [3, 4] or its improved technologies. 2.2 Computing After image preprocessing, the point cloud can be computed based on the principle of camera imaging. As the value of the pixel in the depth image obtained by the preprocessing indicates the depth information of the pixel, which is the linear distance between the surface of the object and the camera (measured in millimeters). It can be computed that the world coordinate system and the image pixel coordinate system have the following conversion relationship shown as Formula (1). ⎡ ⎤ ⎡ ⎤ x u  ⎢ y ⎥ ⎥ z⎣ v ⎦ = K R t ⎢ ⎣z⎦ 1 1

(1)

174

Q. Mi and T. Gao

⎡

⎤ fu s u0 K = ⎣ 0 fv v0 ⎦ 0 0 1

(2)

In Formula (1), x, y, and depth value z are the corresponding coordinates in the world coordinate system of the point (u, v) of the image plane. The intrinsic matrix K (defined as Formula (2)) is related to focal length fu and fv , principal point offset u0 and v0 , and axis skew s. For these parameters are only related to the internal structure of the camera, K is called the camera internal parameter matrix. Align the camera origin and the world origin. That is, [Rt] = [I 0], s is 0, and the image sensor is centered. As a result, x and y can be determined by Formula (3) in the case of the known K value.

x = uz fu (3) y = vz fv

2.3 Registration It is essential to analyze the point cloud computed from the depth image. The translation vector and rotation matrix need to be solved to integrate the obtained point cloud into a unified coordinate system properly through rigid transformation to improve the accuracy and efficiency of the 3D depth image reconstruction with different visual angles. Fast and accurate point cloud registration will improve the reconstruction speed of the 3D model, as well as having a great influence on the fineness and global effect of the final model. Therefore, the performance and applicability of the registration algorithm must be improved. The registration methods are classified into rough registration, fine registration, etc. with different data conditions input and reconstruction requirements output. Coarse Registration. In the case where the relative pose of the point cloud is completely unknown, it is necessary to register the point cloud coarsely first to derive the initial transformation for fine registration. Brute Force (BF) is a more common automatic coarse registration algorithm used for the point cloud. However, due to its high time complexity, the registration based on point cloud feature matching is effective to speed up the search process for different scene characteristics. Fine Registration. Fine registration is a further method of registration for coarse registration results. On the basis of the initial value of the translation vector and rotation matrix after the coarse registration, a more accurate registration result is obtained through the fine registration process which is continuously iteratively converged. Iterative Closest Point (ICP) algorithm [5] is one of the most robust and frequently used method to realize fine registration [6]. After about thirty years of development, variants of ICP algorithm can get registration results faster and more accurately through improvement in different aspects, which is of great significance to the relevant fields of surface registration.

3D Reconstruction Based on the Depth Image: A Review

175

2.4 Fusion During the point cloud registration process, some duplicate data must exist in the overlapping area after the point cloud registered by coordinate transformation to the unified world coordinate system, which leads to data redundancy and inconsistency. Hence, the point cloud data needs to be fused to obtain a more detailed reconstruction model without redundancy. Take the camera’s initial position as the origin to construct a volume grid. The point cloud space will then be divided into a huge amount of small cubes called voxel. A valid approximate simulation of the surface of the object is achieved by assigning Signed Distance Field (SDF) values to all voxels, which is equal to the minimum distance from the voxel to the surface of the reconstructed object. When the SDF value is positive, it indicates that the voxel is in front of the surface. The voxel is behind the surface if the SDF value is negative. Besides, the voxel will be near to the real surface of the object on condition that the SDF value is closer to zero. In fact, it is unnecessary to compute SDF values for all voxels in most cases and Truncated Signed Distance Field (TSDF) [7] algorithm is proposed to solve this. The algorithm only stores several layers of voxels closer to the real surface rather than all voxels, causing it mainly used to solve the problem that voxels occupy a sight of space with great memory consumption reduced. 2.5 Generation As the last step of reconstruction, 3D model surface generation is the key to whether the reconstructed model can be applied practically. The purpose of the surface generation is to construct the visual isosurface of the real object while improving the realism of the model. Marching cubes [8], greedy projection triangulation [9], and Poisson surface reconstruction [10] are classic methods used to complete the generation process.

3 Methods and Discussion Various traditional and novel improved methods exist to achieve 3D reconstruction efficiently with related technologies on the basis of the process of 3D reconstruction based on the depth image in academia and industry. Each algorithm has its own merits. 3.1 Depth Image Preprocessing Image Filtering Algorithms. As a rule, filtering algorithms are classified into linear and nonlinear two types by implementation. Several classical linear and nonlinear filtering algorithms are as follows. Mean Filtering Algorithm. As a common linear filtering algorithm, the mean filtering algorithm is based on the fact that for any pixel in the image, the average of the gray values of neighboring pixels determined by the filtering mask is used instead to eliminate noise interference.

176

Q. Mi and T. Gao

Gaussian Filtering Algorithm. Gaussian filtering is a linear filtering algorithm and it is a process of weighted averaging essentially and it can smooth the edges of the image while denoising. The algorithm outputs a weighted average of neighboring pixels according to the correlation of pixel points in spatial position. The closer the pixel near to the central pixel, the higher the weight. Guided Filtering Algorithm. Guided filtering [11] algorithm is linear which requires the guidance image I and the input image P to obtain the output image Q. The goal of the algorithm is to make P and Q as identical as possible, while the texture portion is similar to I, thereby realizing the effective preservation of the key information of the image. Median Filtering Algorithm. Median filtering is a nonlinear filtering algorithm. The basic principle is to perform sorting in the preset field for the pixels in the image. The median of the sorting is used instead of the original pixel, thus achieving the purpose of eliminating the isolated noise, especially the salt-and-pepper noise. Bilateral Filtering Algorithm. As a nonlinear filtering algorithm, the bilateral filtering [12] algorithm takes into account the spatial positional correlation of the image and the similarity of the pixel values. It can preserve the edge information of the image to a certain extent while suppressing the noise. The weighting coefficient of the bilateral filter consists of two parts. The first part is consistent with the weighting method of the Gaussian filter, which reflects the influence of spatial correlation. And the second one is determined by the difference between the gray value of the neighboring pixel and the center pixel subsequently. Weighted Least Squares Filtering Algorithm. Weighted Least Squares (WLS) filtering [13] algorithm is nonlinear. The algorithm aims to control the spatial scale of the extracted details based on the WLS optimization framework. It has demonstrated the effectiveness on manipulating details at multiple scales, while avoiding visual artifacts. Through the detailed analysis of algorithm characteristics and practical application effects, the advantages and disadvantages of the six algorithms above are given in Table 1 respectively. Table 1. Comparison of the six classical image filtering algorithms. Algorithm

Advantage

Disadvantage

Mean filtering

Reduce sharp changes in image grayscale effectively

Details of the image may be lost to make the image blurry

Gaussian filtering

Perform better in eliminating the normal distribution noise

Edge information of the image may be lost

Guided filtering

Time complexity is independent of the size of the filter window and preserve gradient information

Transferring structural information may result in artifacts (continued)

3D Reconstruction Based on the Depth Image: A Review

177

Table 1. (continued) Algorithm

Advantage

Disadvantage

Median filtering

Perform better in eliminating the isolated noise

The denoise effect is affected by the size of the filter window

Bilateral filtering

Retain the edge information non-iteratively

Halos may appear for color images

WLS filtering

Be suited for progressive coarsening of images and multi-scale detail extraction

Strong manipulations may result in significant changes in the perceived color for color images

To address the disadvantages, many novel improved algorithms have been proposed. Xiao et al. [14] design and implement an efficient hierarchical image weighted mean filtering parallel algorithm for Open Computing Language (OpenCL), which takes full account of the characteristics of image discrete convolution computing. Fu et al. [15] propose a de-hazing algorithm on the basis of the anisotropic Gaussian filtering method to smooth the rough transmission map and improve the clarity of details of the image. A window-aware image filtering framework based on the bilateral filter guided by the local entropy is presented by Liu et al. [16], whose key idea is to design a novel guidance input and a non-box filtering window. Singh et al. [17] design and implement a two-step median filtering anti-forensic framework to hide the median filtering artifacts. The framework provides superior results in terms of image visual quality and forensic undetectability with a better computational time. Patwari et al. [18] introduce a neural network for low dose CT denoising called JBFnet. The filter functions of the Joint Bilateral Filter (JBF) are learned via shallow convolutional networks while the guidance image is estimated by a deep neural network. Ren et al. [19] present a global WLS optimization framework for Polarimetric Synthetic Aperture Radar (PolSAR) despeckling, which can reach a good tradeoff between noise suppressing and detail preservation. Besides, Peng and Huang [20] propose an image denoising method for salt-and-pepper noise, using cascaded filtering based on Overlapped Adaptive Gaussian Smoothing (OAGS) and the Convolutional Refinement Networks (CRNs). The results demonstrate that the proposed method is substantially significant on denoising, especially for high density salt-and-pepper noise. Image Inpainting Algorithms. The filtering algorithms are mainly used for depth image denoising, it cannot be filled correctly and effectively when the depth data has a large area missing. The inpainting algorithms should be used with the filtering algorithms to ensure the effectiveness of depth image preprocessing. Fast Matching Method. FMM and its variants are widely used in image restoration. The method ensures that unknown pixels closest to the known value area are always fixed first. During the inpainting process, the pixels on the edge will be fixed and expanded in layer by layer preferentially until all the points are fixed. Based on FMM, Iwahori et al. [21] proposes an approach to recover 3D shape by using improved FMM. The second-order finite difference and the diagonal grid points are used so that the approach can recover 3D shape more accurately.

178

Q. Mi and T. Gao

Edge-Aware Depth Completion Method. Although FMM has proven its effectiveness in image inpainting, it does not consider the existing depth values for the depth image, which may cause the fixed pixels inaccurate. To solve this, the edge-aware depth completion method [22] is proposed to recover more accurate depth information. The method includes the edge-aware color image analysis and the depth image processing. The reliability of the depth pixel is evaluated first to detect the unreliable region of the depth image. Then the candidate depth propagation starts and the color similarity and distance mapping will be introduced to determine the fix order. The missing depth pixels will be filled by the adjacent depth pixels based on the joint bilateral core finally.

3.2 Point Cloud Registration Iterative Closest Point Algorithm. The essence of ICP algorithm is an optimal matching method based on least squares. It iteratively revises the transformation needed to minimize an error metric, usually the sum of squared differences between the coordinates of the matched pairs, which is the distance from the source to the reference point cloud. In general, the algorithm always converges monotonically to the nearest local minimum of a mean-square distance metric. The rate of convergence during the first few iterations is quite rapid. For the transformation between the source cloud Ps = {s1 , s2 , · · · , sn } and the target cloud Pt = {t1 , t2 , · · · , tn } is rigid, the registration process can be described as Formula (4) to solve the optimal rotation matrix R and translation vector t. R∗ , t ∗ = argmin R,t

|P |

s 1  ti − R · si − t2 |Ps |

(4)

i=1

The general process of ICP algorithm is as follows. Step 1. Match the closest point in the target cloud for each point in the source cloud. Step 2. Weighting points and rejecting outliers. Step 3. Compute the centroid of the source cloud and the target cloud, and convert the two clouds to the centroid coordinate system. Step 4. Get current optimal transformation by using Singular Value Decomposition (SVD) to align each source point to its match found best. Step 5. Compare the variation of the obtained transformation with the criteria for stopping the iterations. Stop iterating if the former is less than the latter, otherwise return to Step 1 and continue the process. Variants of ICP Algorithm. ICP algorithm does not require any local feature extraction or derivative estimation and well suited for coarse-grain or fine-grain parallel architectures. It is relatively insensitive to minor data segmentation errors with a good scalability. However, the limit of ICP algorithm is that it only reach local minima. That is, the result may converge to a false result if most of the nearest points lie in the false direction of the true correspondent points. Besides, the computational overhead of the algorithm is relatively high and it is susceptible to initial transformation and outliers. It only takes into account the point-to-point distance, and the use of point cloud structure information

3D Reconstruction Based on the Depth Image: A Review

179

is lacking. So, various algorithms appear based on the classic ICP algorithm to improve these issues. Point-to-Line ICP Algorithm. An ICP algorithm variant that uses a point-to-line metric is proposed by Censi [23]. The algorithm uses a closed-form minimization and the results demonstrate that it is more precise and requires less iterations. Moreover, Qingshan and Jun [24] combine Normal Distribution Transform (NDT) with the Point-to-Line ICP Algorithm by coarse registering of point clouds between adjacent frames and correct the result subsequently to realize fast and accurate localization and mapping of automatic point clouds. Point-to-Plane ICP Algorithm. Chen and Medioni [25] use a point-to-plane error metric. The minimization object is the sum of the squared distance between a point and the tangent lane at its correspondence point. At each iteration of the point-to-plane ICP algorithm, the computation of the change of relative pose that gives the minimal error is slow. When the relative orientation between the input planes is small, Low [26] proposes an improved method to approximate the nonlinear optimization problem with a linear least squares one that can be solved more efficiently. Generalized ICP Algorithm. Segal et al. [27] thought over synthetically ICP algorithm and the Point-to-Plane ICP algorithm. The Generalized Iterative Closest Point (GICP) algorithm is designed by modelling locally planar surface structure from two scans. This can be thought of as plane-to-plane as well. The main advantage of GICP algorithm is the allowed addition for outlier terms, measurement noise, and other probabilistic techniques. Normal ICP Algorithm. Differently from ICP algorithm, Normal Iterative Closest Point (NICP) Algorithm [28] takes into account each point together with the local features of the normal and curvature [29]. It takes advantage of the 3D structure around the points for the determination of the data association between point clouds. Furthermore, NICP algorithm is based on a least squares formulation of the alignment, which minimizes an augmented error metric depending on the surface characteristics and point coordinates.

3.3 Point Cloud Fusion Truncated Signed Distance Field Algorithm. TSDF algorithm is a volumetric method for integrating range images that possesses the properties, such as incremental updating, representation of directional uncertainty, robustness in the presence of outliers, ability to fill gaps in the reconstruction, etc. It employs a continuous implicit function D(x), which is the weighted signed distance of each point x to the nearest range surface along the line of sight to the camera. The function is constructed as Formula (5) by combining the signed distance function di (x) and weight function wi (x) obtained from the i-th range image. wi (x)di (x) (5) D(x) = wi (x)

180

Q. Mi and T. Gao

The function is represented on a discrete voxel grid and an isosurface is extracted corresponding to D(x) = 0. The isosurface is proved optimal in the least squares sense under certain assumptions. Improved Methods Based on TDSF. On the basis of TSDF algorithm, an approach for compressing TSDF stored in 3D voxel grids and the corresponding textures is proposed by Tang et al. [30]. The compress method relies on a block-based neural network trained end-to-end to upper bounds the reconstruction error by the voxel size and prevent topological errors. Compared to TSDF algorithm, the distortion for the same bitrate is reduced to some extent. Besides, Kang et al. [31] design and implement a novel 3D reconstruction framework to optimize local alignments against depth occlusions and local variants. The framework can help the surface of the target object reconstructed accurately and robustly from multi-view depth maps.

3.4 Model Surface Generation Marching Cubes Algorithm. The basic premise of the marching cubes algorithm is to divide the input volume into a discrete set of cubes. The algorithm proceeds all voxels through the scalar field in sequence and determine the voxels intersecting the isosurface first by comparing the value at each vertex of the voxels with a given threshold. And then it uses interpolation calculation to solve the intersection of each voxel and the isosurface. The isosurface is formed from the triangles connected by the intersections subsequently. However, due to the existence of ambiguities in the interpolant behavior in the cube faces and interior, the meshes extracted by the marching cubes algorithm present topological issues. Chen and Zhang [32] propose neural marching cubes algorithm to reconstruct local mesh topologies more accurately. The neural network learns local features with limited receptive fields, hence it generalizes well to new shapes and datasets. Greedy Projection Triangulation Algorithm. To handle data from height fields like terrain and range scan, greedy projection triangulation algorithm is a traditional method which suitable well. The points are projected into a local 2D coordinate plane and triangulated in the plane, and the triangular mesh surface will be obtained by the topological connection relationship of the points. The algorithm is single pass and specialized to avoid most of the robustness problems faced by purely geometric methods. Poisson Surface Reconstruction Method. The surface reconstruction from oriented points can be cast as a spatial Poisson problem. The Poisson surface reconstruction method considers all points at once without resorting to heuristic spatial partitioning or blending, result in highly resilient to data noise [10]. It is demonstrated to recover fine detail from noisy real-world scans robustly. A novel incremental Poisson surface reconstruction method based on point clouds and the adaptive octree is proposed by Yu et al. [33]. Large scale point clouds are partitioned into small neighboring blocks in the method. The Poisson equation is solved with boundary constraints to obtain the indicator function and extract the surface mesh. The method addresses the incremental reconstruction issue effectively for scenes with newly arrived points.

3D Reconstruction Based on the Depth Image: A Review

181

In addition to the methods above, Sakai and Yasumura [34] propose a new method with Convolutional Neural Network (CNN) to reconstruct the 3D shape by learning features of a 2D image. The spatial information of the image can be acquired from the pooling layer and the convolutional layer of the CNN and the valuable feature of the image can be selected automatically. The method handles the problems in the previous approaches, especially constraint of the fixed position of the light source [35], huge amount of time to search for similar patches [36], and infeasibility to learn the desirable features for 3D reconstruction of 2D images [37].

4 Conclusion The paper surveys the research status of 3D reconstruction and determines the significant research direction as 3D reconstruction based on the depth image. Certain traditional and novel typical works are detailed elaborated and analyzed in the body of the paper. It can be seen from the review that even if more and more sophisticated algorithms and improvement schemes are applied for the 3D reconstruction process, the methods still have room for improvement, which can drive researchers exploring continuously. In sum, the major research should be concentrated in the following aspects in the future. 1. Most of the current studies are focused on overall reconstruction based on images, it is of concern for restoring specific objects in the scene based on the existing depth image or generated point cloud during the reconstruction process. 2. Optimize the relevant algorithms and methods in time and space complexity, robustness and scalability. 3. Achieve better reconstruction efficiency and final results with advanced hardware performance.

Acknowledgments. This work was supported by National Natural Science Foundation of China under Grant Number: 52130403, Fundamental Research Funds for the Central Universities under Grant Number: N2017003.

References 1. Moons, T., Van Gool, L., Vergauwen, M.: 3D reconstruction from multiple images part 1: Principles. Found Trends® Comput Graph Vis. 4(4), 287–404 (2010) 2. Gomes, L., Bellon, O.R.P., Silva, L.: 3D reconstruction methods for digital preservation of cultural heritage: a survey. Pattern Recognit Lett. 50, 3–14 (2014) 3. Sethian, J.A.: A fast marching level set method for monotonically advancing fronts. Proc. Natl. Acad. Sci. 93(4), 1591–1595 (1996) 4. Telea, A.: An image inpainting technique based on the fast marching method. J. Graph Tools 9(1), 23–34 (2004) 5. Besl, P.J., McKay, N.D.: Method for registration of 3-D shapes. In: Sensor Fusion IV: Control Paradigms and Data Structures, vol. 1611, pp. 586–606. SPIE (1992)

182

Q. Mi and T. Gao

6. Liu, R., Burschka, D., Hirzinger, G.: A novel approach to automatic registration of point clouds. In: 2007 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), pp. 401–404. IEEE (2007) 7. Curless, B., Levoy, M.: A volumetric method for building complex models from range images. In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, pp. 303–312. ACM Press (1996) 8. Lorensen, W.E., Cline, H.E.: Marching cubes: a high resolution 3D surface construction algorithm. Comput. Graph (ACM). 21(4), 163–169 (1987) 9. Gopi, M., Krishnan, S.: A fast and efficient projection-based approach for surface reconstruction. In: Proceedings XV Brazilian Symposium on Computer Graphics and Image Processing, pp. 179–186. IEEE (2000) 10. Kazhdan, M., Bolitho, M., Hoppe, H.: Poisson surface reconstruction. In: Proceedings of the fourth Eurographics Symposium on Geometry Processing, vol. 7 (2006) 11. He, K., Sun, J., Tang, X.: Guided image filtering. In: European Conference on Computer Vision, pp. 1–14. Springer (2010) 12. Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: Sixth International Conference on Computer Vision (IEEE Cat. No. 98CH36271), pp. 839–846. IEEE (1998) 13. Farbman, Z., et al.: Edge-preserving decompositions for multi-scale tone and detail manipulation. ACM Trans Graph. 27(3), 1–10 (2008) 14. Xiao, H., et al.: A parallel algorithm of image mean filtering based on OpenCL. IEEE Access. 9, 65001–65016 (2021) 15. Fu, H., et al.: An anisotropic Gaussian filtering model for image de-hazing. IEEE Access. 8, 175140–175149 (2020) 16. Liu, C., Yang, C., Wang, J.: Window-aware guided image filtering via local entropy. IET Image Process. 15(7), 1459–1470 (2021) 17. Singh, K., Kansal, A., Singh, G.: An improved median filtering anti-forensics with better image quality and forensic undetectability. Multidimension. Syst. Signal Process. 30(4), 1951–1974 (2019). https://doi.org/10.1007/s11045-019-00637-8 18. Patwari, M., et al.: JBFnet-low dose CT denoising by trainable joint bilateral filtering. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), pp. 506–515. Springer (2020) 19. Ren, Y., et al.: A global weighted least-squares optimization framework for speckle filtering of PolSAR imagery. IEEE Trans. Geosci. Remote Sens. 57(3), 1265–1277 (2018) 20. Peng, Y.T., Huang, S.W.: Image impulse noise removal using cascaded filtering based on overlapped adaptive gaussian smoothing and convolutional refinement networks. IEEE Open J. Comput. Soc. 2, 382–392 (2021) 21. Iwahori, Y., et al.: Shape recovery using improved fast marching method for SEM image. In: 2018 5th International Conference on Computational Science/Intelligence and Applied Informatics (CSII), pp. 60–65. IEEE (2018) 22. Huang, Y. L., Hsu, T. W., Chien, S. Y.: Edge-aware depth completion for point-cloud 3D scene visualization on an RGB-D camera. In: 2014 IEEE Visual Communications and Image Processing Conference (VCIP), pp. 422–425. IEEE (2014) 23. Censi, A.: An ICP variant using a point-to-line metric. In: 2008 IEEE International Conference on Robotics and Automation (ICRA), pp. 19–25. IEEE (2008) 24. Qingshan, W., Jun, Z.: Point Cloud Registration Algorithm Based on Combination of NDT and PLICP. In: 2019 15th International Conference on Computational Intelligence and Security (CIS), pp. 132–136. IEEE (2019) 25. Chen, Y., Medioni, G.: Object modelling by registration of multiple range images. Image Vis Comput. 10(3), 145–155 (1992) 26. Low, K.L.: Linear least-squares optimization for point-to-plane icp surface registration. Chapel Hill, University of North Carolina 4(10), 1–3 (2004)

3D Reconstruction Based on the Depth Image: A Review

183

27. Segal, A., Haehnel, D., Thrun, S.: Generalized-icp. In: Robotics: science and systems, vol. 2, No. 4, p. 435 (2009) 28. Serafin, J., Grisetti, G.: Using augmented measurements to improve the convergence of ICP. In: International Conference on Simulation, Modeling, and Programming for Autonomous Robots, pp. 566–577. Springer (2014) 29. Serafin, J.: Using Extended Measurements and Geometric Features for Robust Long-Term Localization and Mapping. Doctoral dissertation, Sapienza University of Rome (2017) 30. Tang, D., et al.: Deep implicit volume compression. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp. 1293–1303. IEEE (2020) 31. Kang, J., et al.: WarpingFusion: accurate multi-view TSDF fusion with local perspective warp. In: 2021 IEEE International Conference on Image Processing (ICIP), pp. 1564–1568. IEEE (2021) 32. Chen, Z., Zhang, H.: Neural marching cubes. ACM Trans Graph. 40(6), 1–15 (2021) 33. Yu, Q., et al.: Incremental poisson surface reconstruction for large scale three-dimensional modeling. In: Chinese Conference on Pattern Recognition and Computer Vision (PRCV), pp. 442–453. Springer (2019) 34. Sakai, K., Yasumura, Y.: Three-dimensional shape reconstruction from a single image based on feature learning. In: 2018 International Workshop on Advanced Image Technology (IWAIT), pp. 1–4. IEEE (2018) 35. Zhang, R., et al.: Shape-from-shading: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 21(8), 690–706 (1999) 36. Hassner, T., Basri, R.: Example based 3D reconstruction from single 2D images. In: 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW’06), p. 15. IEEE (2006) 37. Mori, Y., Yasumura, Y., Uehara, K.: 3D Face reconstruction from a single image using machine learning methodology. In: Proceedings of the 2009 ICMITA, pp. 29–32 (2009)

Mesh Routers Placement by WMN-PSODGA Simulation System Considering Stadium Distribution and RDVM: A Comparison Study for UNDX and UNDX-m Methods Admir Barolli1(B) , Kevin Bylykbashi2 , Ermioni Qafzezi3 , Shinji Sakamoto4 , Leonard Barolli2 , and Makoto Takizawa5 1

2

Department of Information Technology, Aleksander Moisiu University of Durres, L.1, Rruga e Currilave, Durres, Albania [email protected] Department of Information and Communication Engineering, Fukuoka Institute of Technology, 3-30-1 Wajiro-Higashi, Higashi-Ku, Fukuoka 811-0295, Japan [email protected], [email protected] 3 Graduate School of Engineering, Fukuoka Institute of Technology, 3-30-1 Wajiro-Higashi, Higashi-Ku, Fukuoka 811-0295, Japan [email protected] 4 Department of Information and Computer Science, Kanazawa Institute of Technology, 7-1 Ohgigaoka Nonoichi, Ishikawa 921-8501, Japan [email protected] 5 Department of Advanced Sciences, Faculty of Science and Engineering, Hosei University, 3-7-2, Kajino-machi, Koganei-shi, Tokyo 184-8584, Japan [email protected]

Abstract. Wireless Mesh Networks (WMNs) are gaining a lot of attention from researchers due to their advantages such as easy maintenance, low upfront cost, and high robustness. However, designing a robust WMN at low cost requires the use of the least possible mesh routers but still interconnected and able to offer full coverage. Therefore, the placement of mesh routers over the area of interest is a problem that entails thorough planning. In our previous work, we implemented a simulation system that deals with this problem considering Particle Swarm Optimization (PSO) and Distributed Genetic Algorithm (DGA), called WMNPSODGA. In this paper, we compare the results of Stadium distribution of mesh clients for Unimodal Normal Distribution Crossover (UNDX) and Multi-parental UNDX (UNDX-m) methods with Rational Decrement of Vmax Method (RDVM) as a router replacement method. The simulation results show that UNDX achieves full client coverage, better connectivity and improved load balance.

c The Author(s), under exclusive license to Springer Nature Switzerland AG 2022  L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 184–195, 2022. https://doi.org/10.1007/978-3-031-08819-3_18

A Hybrid Intelligent Simulation System for Node Placement in WMNs

1

185

Introduction

The wireless networks and devices are becoming increasingly popular and they provide users access to information and communication anytime and anywhere [2,7,10,18]. Wireless Mesh Networks (WMNs) are gaining a lot of attention because of their low-cost nature that makes them attractive for providing wireless Internet connectivity. A WMN is dynamically self-organized and selfconfigured, with the nodes in the network automatically establishing and maintaining mesh connectivity among itself (creating, in effect, an ad hoc network). This feature brings many advantages to WMN such as low up-front cost, easy network maintenance, robustness and reliable service coverage [1]. Moreover, such infrastructure can be used to deploy community networks, metropolitan area networks, municipal and corporative networks, and to support applications for urban areas, medical, transport and surveillance systems. Mesh node placement in WMNs can be seen as a family of problems, which is shown (through graph theoretic approaches or placement problems, e.g. [5,11]) to be computationally hard to solve for most of the formulations [22]. We consider the version of the mesh router nodes placement problem in which we are given a grid area where to deploy a number of mesh router nodes and a number of mesh client nodes of fixed positions (of an arbitrary distribution) in the grid area. The objective is to find a location assignment for the mesh routers to the cells of the grid area that maximizes the network connectivity, client coverage and consider load balancing for each router. Network connectivity is measured by Size of Giant Component (SGC) of the resulting WMN graph, while the user coverage is simply the number of mesh client nodes that fall within the radio coverage of at least one mesh router node and is measured by Number of Covered Mesh Clients (NCMC). For load balancing, we added in the fitness function a new parameter called NCMCpR (Number of Covered Mesh Clients per Router). Node placement problems are known to be computationally hard to solve [8, 9,23]. In previous works, some intelligent algorithms have been recently investigated for node placement problem [3,6,12,14]. In [16], we implemented a Particle Swarm Optimization (PSO) based simulation system, called WMN-PSO. Also, we implemented another simulation system based on Genetic Algorithm (GA), called WMN-GA [15], for solving node placement problem in WMNs. Then, we designed and implemented a hybrid simulation system based on PSO and distributed GA (DGA). We call this system WMN-PSODGA. In this paper, we compare the results of Unimodal Normal Distribution Crossover (UNDX) and Multi-parental UNDX (UNDX-m) methods considering the Stadium distribution of mesh clients with Rational Decrement of Vmax Method (RDVM) as a router replacement method. The rest of the paper is organized as follows. In Sect. 2, we introduce intelligent algorithms. In Sect. 3 is presented the implemented hybrid simulation system. The simulation results are given in Sect. 4. Finally, we give conclusions and future work in Sect. 5.

186

2 2.1

A. Barolli et al.

Intelligent Algorithms for Proposed Hybrid Simulation System Particle Swarm Optimization

In PSO a number of simple entities (the particles) are placed in the search space of some problem or function and each evaluates the objective function at its current location. The objective function is often minimized and the exploration of the search space is not through evolution [13]. Each particle then determines its movement through the search space by combining some aspect of the history of its own current and best (best-fitness) locations with those of one or more members of the swarm, with some random perturbations. The next iteration takes place after all particles have been moved. Eventually the swarm as a whole, like a flock of birds collectively foraging for food, is likely to move close to an optimum of the fitness function. Each individual in the particle swarm is composed of three D-dimensional vectors, where D is the dimensionality of the search space. These are the current position xi , the previous best position pi and the velocity vi . The particle swarm is more than just a collection of particles. A particle by itself has almost no power to solve any problem; progress occurs only when the particles interact. Problem solving is a population-wide phenomenon, emerging from the individual behaviors of the particles through their interactions. In any case, populations are organized according to some sort of communication structure or topology, often thought of as a social network. The topology typically consists of bidirectional edges connecting pairs of particles, so that if j is in i’s neighborhood, i is also in j’s. Each particle communicates with some other particles and is affected by the best point found by any member of its topological neighborhood. This is just the vector pi for that best neighbor, which we will denote with pg . The potential kinds of population “social networks” are hugely varied, but in practice certain types have been used more frequently. We show the pseudo code of PSO in Algorithm 1. In the PSO process, the velocity of each particle is iteratively adjusted so that the particle stochastically oscillates around pi and pg locations. 2.2

Distributed Genetic Algorithm

Distributed Genetic Algorithm (DGA) has been used in various fields of science. DGA has shown their usefulness for the resolution of many computationally hard combinatorial optimization problems. We show the pseudo code of DGA in Algorithm 2. Population of individuals: Unlike local search techniques that construct a path in the solution space jumping from one solution to another one through local perturbations, DGA use a population of individuals giving thus the search a larger scope and chances to find better solutions. This feature is also known as “exploration” process in difference to “exploitation” process of local search methods.

A Hybrid Intelligent Simulation System for Node Placement in WMNs

187

Algorithm 1. Pseudo code of PSO. /* Initialize all parameters for PSO */ Computation maxtime:= T pmax , t := 0; Number of particle-patterns:= m, 2 ≤ m ∈ N 1 ; Particle-patterns initial solution:= P 0i ; Particle-patterns initial position:= x0ij ; Particles initial velocity:= v 0ij ; PSO parameter:= ω, 0 < ω ∈ R1 ; PSO parameter:= C1 , 0 < C1 ∈ R1 ; PSO parameter:= C2 , 0 < C2 ∈ R1 ; /* Start PSO */ Evaluate(G0 , P 0 ); while t < T pmax do /* Update velocities and positions */ = ω · v tij v t+1 ij +C1 · rand() · (best(Pijt ) − xtij ) +C2 · rand() · (best(Gt ) − xtij ); t+1 xij = xtij + v t+1 ij ; /* if fitness value is increased, a new solution will be accepted. */ Update Solutions(Gt , P t ); t = t + 1; end while Update Solutions(Gt , P t ); return Best found pattern of particles as solution;

Fitness: The determination of an appropriate fitness function, together with the chromosome encoding are crucial to the performance of DGA. Ideally we would construct objective functions with “certain regularities”, i.e. objective functions that verify that for any two individuals which are close in the search space, their respective values in the objective functions are similar. Selection: The selection of individuals to be crossed is another important aspect in DGA as it impacts on the convergence of the algorithm. Several selection schemes have been proposed in the literature for selection operators trying to cope with premature convergence of DGA. There are many selection methods in GA. In our system, we implement 2 selection methods: Random method and Roulette wheel method. Crossover operators: Use of crossover operators is one of the most important characteristics. Crossover operator is the means of DGA to transmit best genetic features of parents to offsprings during generations of the evolution process. Many methods for crossover operators have been proposed such as Blend Crossover (BLX-α), Simplex Crossover (SPX), Unimodal Normal Distribution Crossover (UNDX), and Multi-parental UNDX (UNDX-m). In this paper, we implement and compare the results that the last two methods achieve for the same implementation of the system. Mutation operators: These operators intend to improve the individuals of a population by small local perturbations. They aim to provide a component of

188

A. Barolli et al.

Algorithm 2. Pseudo code of DGA. /* Initialize all parameters for DGA */ Computation maxtime:= T gmax , t := 0; Number of islands:= n, 1 ≤ n ∈ N 1 ; initial solution:= P 0i ; /* Start DGA */ Evaluate(G0 , P 0 ); while t < T gmax do for all islands do Selection(); Crossover(); Mutation(); end for t = t + 1; end while Update Solutions(Gt , P t ); return Best found pattern of particles as solution;

randomness in the neighborhood of the individuals of the population. In our system, we implemented two mutation methods: uniformly random mutation and boundary mutation. Escaping from local optima: GA itself has the ability to avoid falling prematurely into local optima and can eventually escape from them during the search process. DGA has one more mechanism to escape from local optima by considering some islands. Each island computes GA for optimizing and they migrate its gene to provide the ability to avoid from local optima (See Fig. 1). Convergence: The convergence of the algorithm is the mechanism of DGA to reach to good solutions. A premature convergence of the algorithm would cause that all individuals of the population be similar in their genetic features and thus the search would result ineffective and the algorithm getting stuck into local optima. Maintaining the diversity of the population is therefore very important to this family of evolutionary algorithms.

3

Proposed and Implemented WMN-PSODGA Hybrid Intelligent Simulation System

In this section, we present the proposed WMN-PSODGA hybrid intelligent simulation system. In the following, we describe the initialization, particle-pattern, gene coding, fitness function, and replacement methods. The pseudo code of our implemented system is shown in Algorithm 3. Also, our implemented simulation system uses Migration function as shown in Fig. 2. The Migration function swaps solutions among lands included in PSO part.

A Hybrid Intelligent Simulation System for Node Placement in WMNs

189

Fig. 1. Model of Migration in DGA.

Algorithm 3. Pseudo code of WMN-PSODGA system. Computation maxtime:= Tmax , t := 0; Initial solutions: P . Initial global solutions: G. /* Start PSODGA */ while t < Tmax do Subprocess(PSO); Subprocess(DGA); WaitSubprocesses(); Evaluate(Gt , P t ) /* Migration() swaps solutions (see Fig. 2). */ Migration(); t = t + 1; end while Update Solutions(Gt , P t ); return Best found pattern of particles as solution;

Initialization We decide the velocity of particles by a random process considering the area size. For√instance, when √ the area size is W × H, the velocity is decided randomly from − W 2 + H 2 to W 2 + H 2 . Particle-Pattern A particle is a mesh router. A fitness value of a particle-pattern is computed by combination of mesh routers and mesh clients positions. In other words, each particle-pattern is a solution as shown is Fig. 3.

190

A. Barolli et al.

Fig. 2. Model of WMN-PSODGA migration.

Fig. 3. Relationship among global solution, particle-patterns, and mesh routers in PSO part.

Gene Coding A gene describes a WMN. Each individual has its own combination of mesh nodes. In other words, each individual has a fitness value. Therefore, the combination of mesh nodes is a solution. Fitness Function WMN-PSODGA has the fitness function to evaluate the temporary solution of the router’s placements. The fitness function is defined as: F itness = α × N CM C(xij , y ij ) + β × SGC(xij , y ij ) + γ × N CM CpR(xij , y ij ). This function uses the following indicators. • NCMC (Number of Covered Mesh Clients) The NCMC is the number of the clients covered by the SGC’s routers. • SGC (Size of Giant Component) The SGC is the maximum number of connected routers.

A Hybrid Intelligent Simulation System for Node Placement in WMNs

191

• NCMCpR (Number of Covered Mesh Clients per Router) The NCMCpR is the number of clients covered by each router. The NCMCpR indicator is used for load balancing. WMN-PSODGA aims to maximize the value of the fitness function in order to optimize the placements of the routers using the above three indicators. Weight-coefficients of the fitness function are α, β, and γ for NCMC, SGC, and NCMCpR, respectively. Moreover, the weight-coefficients are implemented as α + β + γ = 1. Router Replacement Methods A mesh router has x, y positions, and velocity. Mesh routers are moved based on velocities. There are many router replacement methods. In this paper, we consider the Rational Decrement of Vmax Method (RDVM). Constriction Method (CM) CM is a method which PSO parameters are set to a week stable region (ω = 0.729, C1 = C2 = 1.4955) based on analysis of PSO by M. Clerc et al. [4,20]. Random Inertia Weight Method (RIWM) In RIWM, the ω parameter is changing ramdomly from 0.5 to 1.0. The C1 and C2 are kept 2.0. The ω can be estimated by the week stable region. The average of ω is 0.75 [20]. Linearly Decreasing Inertia Weight Method (LDIWM) In LDIWM, C1 and C2 are set to 2.0, constantly. On the other hand, the ω parameter is changed linearly from unstable region (ω = 0.9) to stable region (ω = 0.4) with increasing of iterations of computations [20,21]. Linearly Decreasing Vmax Method (LDVM) In LDVM, PSO parameters are set to unstable region (ω = 0.9, C1 = C2 = 2.0). A value of Vmax which is maximum velocity of particles is considered. With increasing of iteration of computations, the Vmax is kept decreasing linearly [19]. Rational Decrement of Vmax Method (RDVM) In RDVM, PSO parameters are set to unstable region (ω = 0.9, C1 = C2 = 2.0). The Vmax is kept decreasing with the increasing of iterations as Vmax (x) =



W 2 + H2 ×

T −x . x

Where, W and H are the width and the height of the considered area, respectively. Also, T and x are the total number of iterations and a current number of iteration, respectively [17].

192

A. Barolli et al. Table 1. The common parameters for each simulation. Parameters

Values

Distribution of Mesh Clients Stadium Number of Mesh Clients

48

Number of Mesh Routers

17

Radius of a Mesh Router

2.0–3.5

Number of GA Islands

16

Number of Migrations

200

Evolution Steps

9

Selection Method

Random Method

Crossover Method

UNDX, UNDX-m

Mutation Method

Uniform Mutation

Crossover Rate

0.8

Mutation Rate

0.2

Replacement Method

RDVM

Area Size

32.0 × 32.0

Fig. 4. Visualization results after the optimization.

4

Simulation Results

In this section, we present and compare the simulation results of UNDX and UNDX-m considering Stadium distribution of mesh clients and RDVM as a router replacement method. The weight-coefficients of fitness function were adjusted for optimization. In this paper, the weight-coefficients are α = 0.8, β = 0.1, γ = 0.1. The number of mesh routers and mesh clients is 17 and

A Hybrid Intelligent Simulation System for Node Placement in WMNs 7

Number of Covered Clients

Number of Covered Clients

6 5 4 3 2 1 0

193

6 5 4 3 2 1 0

[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16]

[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16]

Router

Router

(a) UNDX

(b) UNDX-m

Fig. 5. Number of covered clients by each router after the optimization. 3

regression line data

r = -0.700657 Standard Deviation

Standard Deviation

3

2

1

0

10

20

30

40

50

Number of Updates

(a) UNDX

60

70

80

regression line data

r = -0.334927

2

1

0

5

10

15

20

25

30

35

Number of Updates

40

45

50

(b) UNDX-m

Fig. 6. Transition of the standard deviations.

48, whereas selection and mutation methods are Random and Uniform, respectively. Table 1 summarizes the common parameters used for simulations. Figure 4 shows the visualization results after the optimization, Fig. 5 the number of covered mesh clients by each route, whereas Fig. 6 the standard deviation where r is the correlation coefficient. As shown in Fig. 4(a), the simulation results show that when using UNDX the mesh routers cover all mesh clients and enable better connectivity by forming a ring topology among mesh routers. On the other hand, Fig. 4(b), when using UNDX-m the mesh routers do not cover all mesh clients. In Fig. 5(a), Fig. 5(b), Fig. 6(a) and Fig. 6(b), we see the results in terms of load balancing. We can see which case has better results by comparing their standard deviations and their correlation coefficients. When the standard deviation is an increasing line (r > 0), the number of mesh clients for each router tends to be different. On the other hand, when the standard deviation is a decreasing line (r < 0), the number of mesh clients for each router tends to go close to each other. The standard deviation is a decreasing line in both cases, but a better load balancing is achieved when UNDX method is used.

194

5

A. Barolli et al.

Conclusions

In this work, we evaluated the performance of WMNs using a hybrid simulation system based on PSO and DGA (called WMN-PSODGA). We compared the simulation results of Stadium distribution of mesh clients and RDVM router replacement method for UNDX and UNDX-m methods. The simulation results show that UNDX achieves full client coverage, better connectivity and improved load balance. Acceptable connectivity and load balancing are achieved when using UNDX-m, too, but not all mesh clients are covered in this case. In future work, we will consider other crossover and mutation methods.

References 1. Akyildiz, I.F., Wang, X., Wang, W.: Wireless mesh networks: a survey. Comput. Netw. 47(4), 445–487 (2005) 2. Barolli, A., Sakamoto, S., Barolli, L., Takizawa, M.: Performance analysis of simulation system based on particle swarm optimization and distributed genetic algorithm for WMNS considering different distributions of mesh clients. In: Barolli, L., Xhafa, F., Javaid, N., Enokido, T. (eds.) IMIS 2018. AISC, vol. 773, pp. 32–45. Springer, Cham (2019). https://doi.org/10.1007/978-3-319-93554-6 3 3. Barolli, A., Sakamoto, S., Ozera, K., Barolli, L., Kulla, E., Takizawa, M.: Design and implementation of a hybrid intelligent system based on particle swarm optimization and distributed genetic algorithm. In: Barolli, L., Xhafa, F., Javaid, N., Spaho, E., Kolici, V. (eds.) EIDWT 2018. LNDECT, vol. 17, pp. 79–93. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75928-9 7 4. Clerc, M., Kennedy, J.: The particle swarm-explosion, stability, and convergence in a multidimensional complex space. IEEE Trans. Evol. Comput. 6(1), 58–73 (2002) 5. Franklin, A.A., Murthy, C.S.R.: Node placement algorithm for deployment of twotier wireless mesh networks. In: Proceedings of Global Telecommunications Conference, pp. 4823–4827 (2007) 6. Girgis, M.R., Mahmoud, T.M., Abdullatif, B.A., Rabie, A.M.: Solving the wireless mesh network design problem using genetic algorithm and simulated annealing optimization methods. Int. J. Comput. Appl. 96(11), 1–10 (2014) 7. Goto, K., Sasaki, Y., Hara, T., Nishio, S.: Data gathering using mobile agents for reducing traffic in dense mobile wireless sensor networks. Mob. Inf. Syst. 9(4), 295–314 (2013) 8. Lim, A., Rodrigues, B., Wang, F., Xu, Z.: k-center problems with minimum coverage. Theoret. Comput. Sci. 332(1–3), 1–17 (2005) 9. Maolin, T., et al.: Gateways placement in backbone wireless mesh networks. Int. J. Commun. Netw. Syst. Sci. 2(1), 44–50 (2009) 10. Matsuo, K., Sakamoto, S., Oda, T., Barolli, A., Ikeda, M., Barolli, L.: Performance analysis of WMNs by WMN-GA simulation system for two WMN architectures and different TCP congestion-avoidance algorithms and client distributions. Int. J. Commun. Networks Distrib. Syst. 20(3), 335–351 (2018) 11. Muthaiah, S.N., Rosenberg, C.P.: Single gateway placement in wireless mesh networks. In: Proceedings of 8th International IEEE Symposium on Computer Networks, pp. 4754–4759 (2008)

A Hybrid Intelligent Simulation System for Node Placement in WMNs

195

12. Naka, S., Genji, T., Yura, T., Fukuyama, Y.: A hybrid particle swarm optimization for distribution state estimation. IEEE Trans. Power Syst. 18(1), 60–68 (2003) 13. Poli, R., Kennedy, J., Blackwell, T.: Particle swarm optimization. Swarm Intell. 1(1), 33–57 (2007) 14. Sakamoto, S., Kulla, E., Oda, T., Ikeda, M., Barolli, L., Xhafa, F.: A comparison study of simulated annealing and genetic algorithm for node placement problem in wireless mesh networks. J. Mob. Multimedia 9(1–2), 101–110 (2013) 15. Sakamoto, S., Kulla, E., Oda, T., Ikeda, M., Barolli, L., Xhafa, F.: A comparison study of hill climbing, simulated annealing and genetic algorithm for node placement problem in WMNs. J. High Speed Networks 20(1), 55–66 (2014) 16. Sakamoto, S., Oda, T., Ikeda, M., Barolli, L., Xhafa, F.: Implementation and evaluation of a simulation system based on particle swarm optimisation for node placement problem in wireless mesh networks. Int. J. Commun. Networks Distribut. Syst. 17(1), 1–13 (2016) 17. Sakamoto, S., Oda, T., Ikeda, M., Barolli, L., Xhafa, F.: Implementation of a new replacement method in WMN-PSO simulation system and its performance evaluation. In: The 30th IEEE International Conference on Advanced Information Networking and Applications (AINA-2016), pp. 206–211 (2016) 18. Sakamoto, S., Ozera, K., Ikeda, M., Barolli, L.: Implementation of intelligent hybrid systems for node placement problem in WMNs considering particle swarm optimization, hill climbing and simulated annealing. Mob. Networks Appl. 23(1), 27–33 (2017). https://doi.org/10.1007/s11036-017-0897-7 19. Schutte, J.F., Groenwold, A.A.: A study of global optimization using particle swarms. J. Global Optim. 31(1), 93–108 (2005) 20. Shi, Y.: Particle swarm optimization. IEEE Connect. 2(1), 8–13 (2004) 21. Shi, Y., Eberhart, R.C.: Parameter Selection in particle swarm optimization. In: Evolutionary programming VII, pp. 591–600 (1998) 22. Vanhatupa, T., Hannikainen, M., Hamalainen, T.: Genetic algorithm to optimize node placement and configuration for WLAN planning. In: Proceedings of The 4th IEEE International Symposium on Wireless Communication Systems, pp. 612–616 (2007) 23. Wang, J., Xie, B., Cai, K., Agrawal, D.P.: Efficient mesh router placement in wireless mesh networks. In: Proceedings of IEEE International Conference on Mobile Ad-hoc and Sensor Systems (MASS-2007). pp. 1–9 (2007)

Unsupervised Deep Image Set Hashing for Efficient Multi-label Image Retrieval Nan Guo(B) , Cuixia Bai, and Yunxia Yang Computer Science and Engineering College, Northeastern University, Shenyang 110004, China [email protected], {1971575,2101838}@stu.neu.edu.cn

Abstract. Hashing has been widely used in approximate nearest neighbor searching for large-scale image retrieval. Recently, many deep hashing methods have been proposed and significantly shown improved performance. However, most of these methods encode each image separately, ignoring that the images in the same set represent the same object or person. When used in large-scale scenes, they need to be compared one by one, which leads to poor retrieval performance. To solve this problem, we propose an unsupervised deep image set hashing method for multilabel image retrieval, which takes the image set as input, extracts image features and calculates set features, then constructs semantic structure through statistical information of features, and finally preserves this semantic structure through pair-wise loss function. Experiments show that our method has certain advantages on benchmark datasets.

1

Introduction

The development of multimedia technology has generated massive image resources. How to retrieve the required images efficiently for fast and convenient web services has become a research hotspot in computer vision. Approximate Nearest Neighbor (ANN) search [1] is often used for indexing and matching of the image features. Among ANN search methods, the hash-based methods map high-dimensional features to low-dimensional binary features, which can search for the required images in constant time and is more suitable for large-scale datasets. The traditional hashing methods [2–5] use hash collisions to make hash code distribution close to the original data distribution. With the progress of deep learning, it is a trend to combine deep learning with hashing. Compared with traditional hashing methods, deep hashing methods [6–18] have better performance. However, many early deep hashing methods were designed for single-label images, and images are often associated with multiple topics in fact. Therefore, multi-label image retrieval has received extensive attention. But multi-label image retrieval methods always encode a single image and ignore the similarity between images. c The Author(s), under exclusive license to Springer Nature Switzerland AG 2022  L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 196–206, 2022. https://doi.org/10.1007/978-3-031-08819-3_19

Unsupervised Deep Image Set Hashing

197

Inspired by the deep set hashing in single-label image retrieval [18], we propose a new unsupervised deep image set hashing model for multi-label image retrieval, which takes the image set as input and learns a unique hash code to represent a group of similar images, it can reduce the complexity and time cost for image retrieval and matching. Experiments show that our method has more efficient retrieval performance. The main contributions of this paper can be summarized as follows: 1. We propose an unsupervised deep image set hashing method, which takes image set as the input of multi-label image retrieval. 2. We use statistics to calculate the set features, and then we use the distribution information of set features to construct a semantic structure for unsupervised learning. 3. Experiments show that compared with the classical hashing methods, the proposed deep hashing method based on image set obtains more efficient performance, it can be seen that the method can effectively improve retrieval MAP and shorten the comparison time in retrieval. The remainder of this paper is organized as follows: we discuss the related work in Sect. 2, introduce the approach in Sect. 3, Sect. 4 is the experimental results, and Sect. 5 is a summary of the paper.

2

Related Work

In recent years, hashing methods have been widely used because of their efficient computing and storage capacity. Traditional hashing methods for image retrieval mainly improve the retrieval performance by improving hash function and similarity metrics. LSH [2] maps several hash tables by hash functions, ensuring that samples with high similarity can be mapped to the same hash bucket with a certain probability. SH [3] is closely related to graph partitioning problem, which is NP-hard. SpH [4] maps more coherent data points in space into a binary code, in which a new spherical Hamming distance is designed. ITQ [5] uses an alternating minimization scheme to find the rotation of zero-centered data. Although these methods have made some progress, they usually rely on pre-defined features and are difficult to represent rich semantic information, so the retrieval performance can not be further improved. Deep Hashing for Single-Label Image Retrieval. With deep learning shining brilliantly in artificial intelligence, deep hashing has achieved promising results. For supervised methods, CNNH [6] combines CNN with hashing for the first time. It is a two-stage method, and separates the approximate hash code learning from the image feature representation process, so the retrieval efficiency is not high. After that, DNNH [7] and DLBHC [8] both adopted onestage method, and designed a deep hash network for learning image features and hash coding at the same time. The difference is that DNNH uses the triplet as the input of the network, while DLBHC uses pair of points as the input.

198

N. Guo et al.

HashNet [9] replaces symbolic function with a smooth activation function in the network, thus solving the ill-conditioned gradient problem. For unsupervised deep methods, DeepBit [10] learns a concise binary code with discrimination without label information. However, it doesn’t further capture the high-level semantic relationship between different data points. SSDH [11] learns the hashing function by minimizing the distance between the semantic structure and the similarity structure of hash codes obtained through the network in the loss stage. Different from SSDH, Tu et al. [12] use the intrinsic manifold structure in feature space and cosine similarity of data points to construct local semantic similarity structure and then use it as guiding information to generate compact hash code. Most methods take a single image as the input and then encode it, but these methods don’t consider that multiple images in a set represent the same class of objects or concepts. The method based on image set has achieved good performance. Feng et al. [18] propose a new deep neural network model, which considered generating binary hash codes with image set as the input and optimizing them so that sets from the same class have smaller distances, while sets from different classes have larger distances. Deep Hashing for Multi-label Image Retrieval. In the real world, the content expressed by images may be associated with multiple labels, so image search based on multiple labels has been widely studied in recent years. DBE [13] uses extended cross entropy to capture the symbiotic dependencies between labels while maintaining the independence of each label. DMLH [14] minimizes the Euclidean distance between the generated binary code and semantic vector for multi-label image retrieval training. DMSSPH [15] and IDHN [16] both consider different degrees of similarity, divide the similarity into layers, and consider the influence of this factor in the loss stage. Chen et al. [17] reduce the hash learning problem in multi-label image retrieval to multi-instance ranking learning problem and propose DMIRH based on multi-instance ranking.

3

Proposed Approach

In this section, we will introduce our approach in detail, including network architecture, set feature computation, semantic structure construction, and hash code learning. 3.1

Network Architecture

In this paper, we define S = {I1 , I2 , · · · , IN } to represent an image set. Ii represents i-th image in the set. mean, var, min and max are the distribution information of the set. Semantic structure D and similarity structure H represent two matrixes, Dij and Hij are used to represent the (i,j)-th element of D and H, respectively.

Unsupervised Deep Image Set Hashing

199

Fig. 1. Network architecture.

Figure 1 shows the network architecture of our model. The model includes several main steps: feature extraction, set feature computation, semantic structure construction and hash code learning. A group of images with similar or identical labels can represent the same objects or concepts [18], and the times of comparisons in the retrieval process can be reduced by using image sets, so we use image set as the input of the model(Image set construction is described in Sect. 4.1). In recent years, the VGG-F network, with five convolution layers (conv1– conv5) and three full connection layers (fc6–fc8), has been applied in many hashing methods [11,16,17] and achieved good performance. Therefore, in the step of feature extraction, we initialize the parameters of the network with the VGG-F model pre-trained on ImageNet, and we can extract high-dimensional feature from the fc7 layer of the VGG-F network, the feature contains rich semantic information and can be used to calculate set features. For the set of data points, statistical data can be used to describe the global characteristics of points distribution well, so in the step of set feature computation, we use common statistical information to calculate set features, which are then used to calculate the hash code and semantic structure D. In supervised learning, most methods use supervised information to construct the semantic structure and learn hash codes to keep semantic similarity or dissimilarity. However, there is no available label information in unsupervised learning. [11,19] shows that the feature extracted from the pre-trained deep network architecture contains rich semantic information, so we construct a semantic structure D by the distribution information of set features as the basis for hash code learning. At the same time, we add a full connection layer with K hidden units, construct a similarity structure H by hash codes, and adjust the network parameters backward through the differences between D and H, so as to achieve the purpose of hash code learning.

200

3.2

N. Guo et al.

Set Feature Computation

Given an input image set S, we need to use an appropriate method to represent it as a unique set feature, while retaining as much information from different images as possible. Inspired by [18], we use the underlying distribution information of set data to describe set characteristics. For each set S, we can calculate various features: the average feature mean(S), the variance var(S), the minimum min(S) and the maximum max(S).

Fig. 2. Illustration of features in sets.

Figure 2 is an illustration of statistical characteristics for sets. It can be seen from the figure that we can use statistical information to model the sets compactly and make the distances farther between the data points which are in different sets, mean(S) is the average feature of the set in the sample space, and var(S) represents the change of the feature related to its average value in a given set, min(S) and max(S) represent the range of the features in the set. The final output of the set computation layer for set S is the direct concatenation of the set statistical characteristics, F (S) = concatenation(mean(S), var(S), min(S), max(S)). 3.3

Semantic Structure Construction

Inspired by [11,19], we extract features from the pre-trained network and analyze the data by statistics. Based on this analysis, we can construct a semantic structure for obtaining the semantic relationship between different data points, which can be used as reference information for hash code learning. Taking the image set based on NUS-WIDE dataset as an example, we use the pre-trained VGG-F network to extract the features of each image in the set and calculate the set features. Then we calculate the cosine distance between each pair of data points, which can be used to measure the similarity between two vectors. The closer the cosine distance is to 1, the more similar the data points are, while the closer the cosine distance is to 0, the less similar they are. Figure 3(a) shows the distance distribution histogram of the cosine distance of all data points, and the original labels of the sets all come from the same group of popular labels. Therefore, the cosine distance of most data is relatively large, and the distance distribution histogram is similar to the normal distribution. Accordingly, we divide the distribution histogram into the left and right parts by

Unsupervised Deep Image Set Hashing

201

using the maximum value, then calculate the mean μlef t and μright , the standard deviation σlef t and σright , respectively, and fit the curve into two appropriate normal distribution curves to obtain Fig. 3(b) and (c).

Fig. 3. Statistics of cosine distance. (a) Distance distribution histogram. (b) Normal distribution of the left part. (c) Normal distribution of the right part.

We define the distance thresholds d1 = μlef t − ασlef t and d2 = μright + βσright , and then divide the distribution into three segments according to the thresholds to construct the semantic structure D. Dij

⎧ ⎨ −1, d(i, j) ≤ d1 0, d1 4 ≥ 4|F |, causing a contradiction. – Subcase 1.2: n ≥ 5. It follows from Lemma 5 that κ2 (Qn ) = 3n − 5 ≤ |NQn ({x, y, z})|. By Lemma 4,we have |NQn ({x, y, z}) ∩ S| ≤ 3 for each S ∈ F . As NQn ({x, y, z}) ⊆ S∈F S, the cardinality of F needs to be |N ({x,y,z})|  ≥ 3n−5 at least Qn 3 3  = n − 1, contradicting the assumption that |F | ≤ n − 3. • Case 2: |V (C)| ≥ 4. Thus, F includes at most |V (P4 )|×|F | = 4|F | ≤ 4(n−3) = 4n − 12 nodes. – Subcase 2.1: n = 4. Then |F | ≤ 1. By Lemma 5, we have κ3 (Q4 ) = 6 > 4 ≥ 4|F |, causing a contradiction. – Subcase 2.2: n = 5. Then |F | ≤ 2. By Lemma 5, we have κ3 (Q5 ) = 10 > 8 ≥ 4|F |, causing a contradiction. – Subcase 2.3: n ≥ 6. By Lemma 5, we have κ3 (Qn ) = 4n − 9 > 4n − 12 ≥ 4|F |, causing a contradiction. To avoid any probable contradiction, it is needed that |F | ≥ n − 2. Next, we show that κ2 (Qn |P4 ) ≤ n − 2 to complete the proof. Suppose that v is any node of Qn . For 1 ≤ i ≤ n − 2, let Si = {(v)i+1 , ((v)0 )i+1 , (((v)0 )1 )i+1 , ((v)1 )i+1 }. Since ((((v)0 )1 )i+1 )0 = ((v)1 )i+1 for 1 ≤ i ≤ n − 2, (v)i+1 , ((v)0 )i+1 , (((v)0 )1 )i+1 , ((v)1 )i+1  is a path of order four. As NQn ({v, (v)0 , ((v)0 )1 , (v)1 }) n−2 n−2 ⊆ i=1 Si , Qn − i=1 Si is disconnected, and Qn [{v, (v)0 , ((v)0 )1 , (v)1 }] is the n−2 smallest component in Qn − i=1 Si . Thus, {S1 , S2 , . . ., Sn−2 } is really a 2-extra   P4 -cut of Qn so that κ2 (Qn |P4 ) ≤ |{S1 , S2 , . . . , Sn−2 }| = n − 2.

264

4

T.-L. Kung and Y.-H. Teng

Derivation of κ2 (Qn |Pk )

It is trivial that κ2 (Qn |P1 ) = κ2 (Qn ), which can be determined by the following Lemma 6. Lemma 6. [15] For 0 ≤ h ≤ n − 2, κh (Qn ) = 2h (n − h). In this section we study κ2 (Qn |P2 ), κ2 (Qn |P3 ) and κ2 (Qn |P4 ). Theorem 4. For n ≥ 4, κ2 (Qn |P2 ) = 2n − 4. Proof. Suppose that v is any node of Qn . For 1 ≤ i ≤ n − 2, let Si = {{(v)i+1 , ((v)0 )i+1 }, {((v)1 )i+1 , (((v)1 )0 )i+1 )}}. n−2  n−2  As NQn ({v, (v)0 , (v)1 , ((v)1 )0 }) ⊆ C∈Si C, Qn − C∈Si C is i=1 i=1 disconnected. Because Qn [{v, (v)0 , (v)1 , ((v)1 )0 }] is the smallest component n−2 n−2 in Qn − i=1 Si , i=1 Si is a 2-extra P2 -cut of Qn . This concludes that n−2 n−2   κ2 (Qn |P2 ) ≤ | i=1 Si | = i=1 |Si | = 2(n − 2) = 2n − 4. Theorem 5. For n ≥ 5, κ2 (Qn |P3 ) ≤

 3n−6 2 3n−5 2

if n is even if n is odd.

Proof. Suppose that v is any node of Qn . For 1 ≤ p ≤  n−2 2 , let Sp = { {((v)0 )2p−1 , (((v)0 )n−1 )2p−1 , ((v)n−1 )2p−1 }, {(v)2p−1 , ((v)2p−1 )2p , (v)2p }, {((v)0 )2p , (((v)0 )n−1 )2p , ((v)n−1 )2p }}.  n−2   • Case 1: n is even. As NQn ({v, (v)0 , ((v)0 )n−1 , (v)n−1 }) ⊂ p=12 C∈Sp C,   n−2  0 0 n−1 2 Qn − , p=1 C∈Sp C is disconnected. Because Qn [{v, (v) , ((v) ) n−2 n−2    2   2  n−1 (v) }] is the smallest component in Qn − p=1 Sp , p=1 Sp is a 2 n−2  restricted P3 -cut of Qn . This concludes that κ2 (Qn |P3 ) ≤ | p=12 Sp | =  n−2 3n−6 2  |Sp | = 3 × n−2 p=1 2 = 2 . n−2 n−2 n−1 • Case 2: n is odd. Thus,  2  = n−3 2 and 2  = 2 . Let S n−1 = { {(v)n−2 , ((v)0 )n−2 , (((v)0 )n−1 )n−2 }, 2

{((v)n−2 )n−1 , (((v)n−2 )n−1 )1 , ((((v)n−2 )n−1 )1 )2 }.  n−2  

 n−2  

2 As NQn ({v, (v)0 , ((v)0 )n−1 , (v)n−1 }) ⊂ p=12 C∈Sp C, Qn − p=1 C∈Sp C is disconnected. Because Qn [{v, (v)0 , ((v)0 )n−1 , (v)n−1 }] is the smallest compo n−2   n−2  nent in Qn − p=12 Sp , p=12 Sp is a 2-restricted P3 -cut of Qn . This concludes  n−2   n−2  3n−5   that κ2 (Qn |P3 ) ≤ | p=12 Sp | = p=12 |Sp | = 3 × n−3 2 +2= 2 .

On the Conditional Pk -connectivity of Hypercube-Based Architectures

265

Theorem 6. For n ≥ 4, κ2 (Qn |P4 ) = n − 2. Proof. The first half of the proof shows that κ2 (Qn |P4 ) ≤ n − 2. Suppose that w is any node of Qn . For 1 ≤ i ≤ n − 2, let Si = {(w)i+1 , ((w)0 )i+1 , (((w)0 )1 )i+1 , ((w)1 )i+1 }. i+1 Since ((((w)0 )1 )i+1 )0 = ((w)1 )i+1 for 1 ≤ i ≤ n − 2, (w) , ((w)0 )i+1 ,  0 1 i+1 1 i+1 (((w) ) ) , ((w) )  is a path of order four. As NQn ( w, (w)0 , )((w)0 )1 ,   n−2 n−2 ⊆ Si , Qn − i=1 Si is disconnected,) and Qn [ w, (w)0 , (w)1 i=1  n−2 ((w)0 )1 , (w)1 ] is the smallest component in Qn − i=1 Si . Thus, {S1 , S2 , . . ., Sn−2 } is a 2-restricted P4 -cut of Qn so that κ2 (Qn |P4 ) ≤ |{S1 , S2 , . . . , Sn−2 }| = n − 2. The second half of the proof argues that κ2 (Qn |P4 ) ≥ n − 2. Suppose that F denotes any 2-restricted P4 -cut of Qn . Then the minimum degree of Qn − F is at least two. Let C denote the component of Qn − F such that δ(C) = δ(Qn − F ). For the sake of contradiction, we assume that |F | ≤ n − 3. Accordingly, F has at most |V (P4 )| × |F | = 4|F | ≤ 4(n − 3) = 4n − 12 nodes.

• Case 1: δ(C) = 2. Because Qn is K3 -free, C includes at least four nodes. – Subcase 1.1: n = 4, 5. By Lemma 5, we have κ3 (Qn ) = n(n−1) > 4n − 12 2 for n = 4, 5, causing a contradiction. – Subcase 1.2: n ≥ 6. It follows from Lemma 5 that κ3 (Qn ) = 4n − 9 > 4n − 12, causing a contradiction. • Case 2: δ(C) ≥ 3. It follows from Lemma 6 that κ3 (Qn ) = 8n − 24 > 4n − 12 for n ≥ 4, which is an immediate contradiction. To avoid each contradiction mentioned above, it is needed that |F | ≥ n − 2.  

5

Conclusion

In this paper, both κ2 (Qn |Pk ) and κ2 (Qn |Pk ) are investigated for k = 2, 3, 4. Regarding to our future work, it is inspired to determine the exact values of both κ2 (Qn |P2 ) and κ2 (Qn |P3 ). Furthermore, it is also intriguing to take k ≥ 5 into consideration. Acknowledgements. This work is supported in part by the Ministry of Science and Technology, Taiwan, under Grant No. MOST 109-2221-E-468-009-MY2.

References 1. Bondy, J.A., Murty, U.S.R.: Graph Theory. Springer, London (2008). https://doi. org/10.1007/978-3-662-53622-3 2. Bossard, A., Kaneko, K.: Cluster-fault tolerant routing in a torus. Sensors 20(11), 1–17 (2020) 3. Dally, W.J., Towles, B.: Principles and Practices of Interconnection Networks. Morgan Kaufmann, San Francisco (2004)

266

T.-L. Kung and Y.-H. Teng

4. Duato, J., Yalamanchili, S., Ni, L.: Interconnection Networks: An Engineering Approach. Morgan Kaufmann, San Francisco (2002) 5. Esfahanian, A.H.: Generalized measures of fault tolerance with application to ncube networks. IEEE Trans. Comput. 38(11), 1586–1591 (1989) 6. F´ abrega, J., Fiol, M.A.: On the extraconnectivity of graphs. Discret. Math. 155, 49–57 (1996) 7. Gu, Q.P., Peng, S.: An efficient algorithm for node-to-node routing in hypercubes with faulty clusters. Comput. J. 39, 14–19 (1996) 8. Gu, Q.P., Peng, S.: k-pairwise cluster fault tolerant routing in hypercubes. IEEE Trans. Comput. 46, 1042–1049 (1997) 9. Gu, Q.P., Peng, S.: Node-to-set and set-to-set cluster fault tolerant routing in hypercubes. Parallel Comput. 24, 1245–1261 (1998) 10. Harary, F., Hayes, J.P., Wu, H.J.: A survey of the theory of hypercube graphs. Comput. Math. Appl. 15, 277–289 (1988) 11. Kung, T.L., Lin, C.K.: Cluster connectivity of hypercube-based networks under the super fault-tolerance condition. Discret. Appl. Math. 293, 143–156 (2021) 12. NASA: Pleiades supercomputer (2021). https://www.nas.nasa.gov/hecc/ resources/pleiades.html 13. Saad, Y., Schultz, M.H.: Topological properties of hypercubes. IEEE Tran. Comput. 37, 867–872 (1988) 14. Sabir, E., Meng, J.: Structure fault tolerance of hypercubes and folded hypercubes. Theor. Comput. Sci. 711, 44–55 (2018) 15. Wu, J., Guo, G.: Fault tolerance measures for m-ary n-dimensional hypercubes based on forbidden faulty sets. IEEE Trans. Comput. 47(8), 888–893 (1998) 16. Yang, W., Meng, J.: Extraconnectivity of hypercubes. Appl. Math. Lett. 22, 887– 891 (2009)

Super K1,p -Connectivity of Locally Twisted Cubes Yuan-Hsiang Teng1 and Tzu-Liang Kung2(B) 1

2

Department of Computer Science and Information Engineering, Providence University, Taichung City, Taiwan [email protected] Department of Computer Science and Information Engineering, Asia University, Taichung City, Taiwan [email protected]

Abstract. Interconnection networks are emerging as an approach to solving system-level communication problems. A network is abstractly modeled by a graph. For p ≥ 1, a p-star K1,p includes p + 1 nodes such that a single node (called center) is linked to each of the other p nodes. The connectivity has long been a classic factor that characterizes both network reliability and fault tolerance. A set F of node subsets of G is a K1,p -cut if G − F is disconnected, and each element of F happens to induce a p-star in G. A super K1,p -cut F of G is a K1,p -cut in G such that the smallest component of G − F contains two or more nodes. Then the super K1,p -connectivity of G, denoted by κ (G|K1,p ), is the cardinality of the minimum super K1,p -cut of G. The locally twisted cube LT Qn is a promising alternative to the hypercube and can serve as the backbone architecture of high-performance computing. In this article, we are inspired to determine κ (LT Qn |K1,p ) for p = 1, 2, 3.

1

Introduction

Interconnection networks are emerging as a solution to the system-level communication problems [4]. A common challenge for network designers is to match the data communication scheme of the problem at hand to the network’s topology. A variety of interconnection networks have been developed on the basis of some well-known undirected graphs, such as meshes, torus, hypercubes, crossed cubes, exchanged hypercubes, butterfly graphs, star graphs, arrangement graphs, Gaussian graphs, etc. [1,5,6,8,20,22,25]. Among the different kinds of network topologies, the hypercube network [10,27] is one of the most attractive candidates for high-performance computing [13,20] due to its promising advantages, for example, including regularity, node-transitivity, edge-symmetry, maximum connectivity, optimal fault-tolerance, Hamiltonicity, and so on. The network’s topological structure is typically modeled by a connected graph for mathematical analysis [2]. A graph G = (V, E) consists of a node set V and an edge set E. For convenience, V (G) and E(G) represent the c The Author(s), under exclusive license to Springer Nature Switzerland AG 2022  L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 267–274, 2022. https://doi.org/10.1007/978-3-031-08819-3_27

268

Y.-H. Teng and T.-L. Kung

node and edge sets of G, respectively. Two nodes u and v of G are neighbors of each other if there is an edge uv joining them in G. For any node v V (G) | uv ∈ E(G)} denotes the neighborhood of v. For of G, NG (v) = {u ∈  S ⊂ V (G), NG (S) = v∈S NG (v) \ S. The minimum degree of G is defined by δ(G) = min{|NG (v)| | v ∈ V (G)}. A graph G is a subgraph of G if V (G ) ⊆ V (G) and E(G ) ⊆ E(G). For any S ⊆ V (G), G[S] denotes the subgraph induced by S. The p-star is a fundamental structure, denoted by K1,p with p ≥ 1. A graph G is connected if for every pair u, v of distinct nodes of G, there exists a path between u and v. A node-cut of G is a subset S of V (G) such that G − S is disconnected or trivial. The connectivity of G is defined as κ(G) = minS⊂V (G) {|S| | G − S is disconnected or trivial}, which is equal to the cardinality of the minimum node-cut of G and can be determined in polynomial time using Menger’s theorem [26]. A graph G is super connected if its minimum node-cut is always composed of a certain node’s neighborhood. A node-cut F of G is a super node-cut if δ(G − F ) > 0. The super connectivity κ (G) of G is defined by κ (G) = min{|F | | F is a super node-cut of G}, i.e., the cardinality of the minimum node-cut if there is any, and by convention, is ∞ otherwise. Super connectivity has been systematically investigated (referring to Chapter 7.7 of [13]). A cluster C in a graph G is a node subsets of G such that G[C] is either connected or trivial. A set F of clusters of G is a cluster-cut if G − F is disconnected or trivial [18]. Suppose that H is any connected graph or trivial graph. Then a cluster-cut F of G is a K1,p -cut if for each cluster C ∈ F , a spanning subgraph of G[C] is isomorphic to K1,p . The K1,p -connectivity of G is denoted by κ(G|K1,p ), which is defined as the cardinality of the minimum K1,p -cut of G. A K1,p -cut F of a connected graph G is a super K1,p -cut if δ(G − F ) > 0. The super K1,p -connectivity κ (G|K1,p ) of G is defined as the cardinality of the minimum super K1,p -cut of G if there is any, and by convention, is ∞ otherwise. Given any connected graph G, κ (G|K1,p ) is greater than or equal to κ(G|K1,p ). A connected graph G is called super K1,p -connected if δ(G − F ) = 0 for every minimum K1,p -cut F of G. Equivalently, G is not super K1,p -connected if κ (G|K1,p ) = κ(G|K1,p ). The locally twisted cube [30] is a hypercube variant with lower diameter than that of the hypercube. Many attractive properties of the locally twisted cube have been widely studied [9,14–16,23,24,28,31]. For example, cycle and path embedding is of high flexibility on locally twisted cubes [15–17]. This article is inspired to determine the super K1,p -connectivity of the locally twisted cube for p = 1, 2, 3. The rest of this paper is structured as follows. Section 2 introduces the topological properties of locally twisted cubes. Section 3 presents the main result of this paper. Finally, some concluding remarks are given in Sect. 4.

2

Preliminary

The node set of the n-dimensional locally twisted cube LT Qn corresponds typically to the set of n-bit binary numbers: LT Q2 is isomorphic to a cycle of

Super K1,p -Connectivity of Locally Twisted Cubes

269

order four, whose node and edge sets are {00, 01, 10, 11} and {(00, 01), (01, 11), (11, 10), (10, 00)}, respectively. For n ≥ 3, LT Qn is a combination of two copies of LT Qn−1 , denoted by LT Q0n−1 and LT Q1n−1 . The node set of LT Q0n−1 is V (LT Q0n−1 ) = {x : 0xn−2 xn−3 · · · x0 | xi ∈ {0, 1} for 0 ≤ i ≤ n − 2}, and the node set of LT Q1n−1 is V (LT Q1n−1 ) = {y : 1yn−2 yn−3 · · · y0 | yi ∈ {0, 1} for 0 ≤ i ≤ n − 2}. Let Ec = {(x, y) | x ∈ V (LT Q0n−1 ), y ∈ V (LT Q1n−1 ), yn−2 = (xn−2 + x0 ) mod 2, yi = xi for every 0 ≤ i ≤ n − 3}. Then the node set of LT Qn is V (LT Qn ) = V (LT Q0n−1 ) ∪ V (LT Q1n−1 ), and the edge set of LT Qn is E(LT Qn ) = E(LT Q0n−1 ) ∪ E(LT Q1n−1 ) ∪ Ec . For the sake of simplification, LT Qn = LT Q0n−1 ⊗ LT Q1n−1 represents the recursive construction of LT Qn . Figure 1 illustrates LT Q3 and LT Q4 . There exists no cycle of order three in LT Qn ; i.e., LT Qn is K3 -free. The n+3 diameter of LT Qn is n+1 2  if n = 3, 4, and 2  if n ≥ 5. Yang et al.. [30] proved that κ(LT Qn ) = n, and some assessments on conditional connectivity of LT Qn have been proposed [3,28]. The locally twisted cube has received a wide variety of researchers’ attention for its attractive properties [9,11,12,14– 17,19,21,23,24,29,30]

Fig. 1. Illustrating LT Q3 and LT Q4 .

For any two adjacent nodes u = un−1 un−2 · · · u0 and v = vn−1 vn−2 · · · v0 in LT Qn , they are the k-neighbors of each other, 0 ≤ k ≤ n − 1, if uk = vk and ui = vi for i > k. For convenience, the k-neighbor of any node v ∈ V (LT Qn ) is denoted by (v)k . It is trivial that ((v)k )k = v. More precisely, (v)0 =  vd−2 · · · v0 , vn−1 · · · v1 v¯0 , (v)1 = vn−1 · · · v2 v¯1 v0 , (v)d = vn−1 · · · vd+1 v¯d vd−1  = (vd−1 + v0 ) mod 2 if 2 ≤ d ≤ n − 2, and (v)n−1 = where vd−1   vn−3 · · · v0 , where vn−2 = (vn−2 + v0 ) mod 2. v¯n−1 vn−2 Apparently, there is no cycle of order three in LT Qn . The following lemma helps locate cycles of order four and five in LT Qn . Lemma 1. [19] Let v = vn−1 vn−2 · · · v0 be any node in LT Qn for n ≥ 3. (i) For 1 ≤ p ≤ n − 2, ((v)p )n−1 = ((v)n−1 )p , and {v, (v)p , (v)n−1 , ((v)p )n−1 } induces a cycle of order four in LT Qn . (ii) If v0 = 0, then (((v)0 )n−1 )0 = ((v)n−1 )n−2 , and {v, (v)0 , (v)n−1 , ((v)0 )n−1 , ((v)n−1 )n−2 } induces a cycle of order five in LT Qn .

270

Y.-H. Teng and T.-L. Kung

(iii) If v1 = 1, then (((v)0 )n−1 )n−2 = ((v)n−1 )0 , and {v, (v)0 , (v)n−1 , ((v)0 )n−1 , ((v)n−1 )0 } induces a cycle of order five in LT Qn . Based on the fact that LT Qn has no cycle of order three, the following lemma holds. Lemma 2. Let {x, y, z} be a set of three nodes in LT Qn such that LT Qn [{x, y, z}] is isomorphic to K1,2 . For any two adjacent nodes, u and v, in LT Qn − {x, y, z}, |NLT Qn ({u, v}) ∩ {x, y, z}| ≤ 2. Lemma 3. [19] For n ≥ 3, let Y = {y0 , y1 , y2 , y3 } ⊂ V (LT Qn ) such that LT Qn [Y ] is isomorphic to K1,3 , where {y1 , y2 , y3 } ⊆ NLT Qn (y0 ). For any two adjacent nodes, u and v, in LT Qn − Y , |Y ∩ NLT Qn ({u, v})| ≤ 3. If y0 ∈ NLT Qn ({u, v}), then |Y ∩ NLT Qn ({u, v})| ≤ 2. The g-extra connectivity of a graph G, κg (G) with g ≥ 0, is the cardinality of the minimum node-cut F of G such that every component of G − F includes at least g + 1 nodes [7]. Lemma 4. [3] For n ≥ 3, κ1 (LT Qn ) = 2n − 2; for n ≥ 5, κ2 (LT Qn ) = 3n − 5; for n ≥ 6, κ3 (LT Qn ) = 4n − 9. The work of Kung et al. [19] summarizes the following lemma. Lemma 5. [19] For n ≥ 3, κ(LT Qn |K1,1 ) = n − 1; κ(LT Qn |K1,2 ) = κ(LT Qn |K1,3 ) = n2 . Lemma 6. [19] For n ≥ 3, LT Qn is not super K1,1 -connected; for n ≥ 4, LT Qn is super K1,2 -connected; for n ≥ 6, LT Qn is super K1,3 -connected.

3

Super K1,p -connectivity of LT Qn

By Lemma 5, κ(LT Qn |K1,1 ) = n − 1 for n ≥ 3. Let x is any node of LT Qn . It follows from Lemma 1 that F = {{(x)i , ((x)1 )i } | 0 ≤ i ≤ n − 1, i = 1} is a minimum K1,1 -cut of LT Qn . Clearly, LT Qn −F consists of two components, and LT Qn [{x, (x)1 }] is the smallest component of LT Qn − F . As δ(LT Qn − F ) = 1, we can determine that κ (LT Qn |K1,1 ) = |F | = n − 1. Theorem 1. For n ≥ 3, κ (LT Qn |K1,1 ) = n − 1. No super K1,2 -cut exists in LT Q3 , and by convention κ (LT Q3 |K1,2 ) = ∞. Lemma 7. For n ≥ 4, κ (LT Qn |K1,2 ) ≤ n − 1. Proof. As LT Qn = LT Q0n−1 ⊗ LT Q1n−1 , let v denote any node of LT Q0n−1 . For 0 ≤ k ≤ n − 1 and k = 1, let ⎧ ⎨ {((v)1 )k , (v)k , ((v)k )2 } if k = 0 Sk = {((v)1 )k , (v)k , ((v)k )k+1 } if 2 ≤ k ≤ n − 2 (1) ⎩ {((v)1 )k , (v)k , ((v)k )0 } if k = n − 1.

Super K1,p -Connectivity of Locally Twisted Cubes

271

By Lemma 1, (v)k and ((v)1 )k are adjacent for each kso that LT Qn [Sk ] is definitely isomorphic to K1,2 . Since NLT Qn ({v, (v)1 }) ⊂ 0≤k≤n−1,k=1 Sk , LT Qn −   0 0≤k≤n−1,k= 1 Sk is disconnected. Let F (0) = V (CQn−1 ) ∩ 0≤k≤n−1,k=1 Sk and F (1) = 0≤k≤n−1,k=1 Sk \ F (0). It is noticed that F (1) = {(v)n−1 , ((v)n−1 )0 , ((v)1 )n−1 , ((v)n−2 )n−1 }. Accordingly, LT Q1n−1 [F (1)] is isomorphic to K1,3 , and CQ1n−1 − F (1) is connected. Because every node of LT Q0n−1 − F (0) − {v, (v)1 } is linked to its (n − 1)-neighbor  in LT Q1n−1 − F (1), LT Qn [{v, (v)1 }] is the smallest component in LT Qn − k=1 Sk . Then F = {S0 , S2 , S3 , . . . , Sn−1 } is a super K1,2 -cut of LT Qn , and the inequality κ (LT Qn |K1,2 ) ≤ |F | = n − 1 holds. Lemma 8. For n ≥ 4, κ (LT Qn |K1,2 ) ≥ n − 1. Proof. Based on Lemma 5, we understand that κ(LT Qn |K1,2 ) = n2 . By Lemma 6, LT Q4 is super K1,2 -connected so that κ (LT Qn |K1,2 ) ≥ κ(LT Q4 |K1,2 ) = 2. Below we consider n ≥ 5. For the sake of contradiction, let F be any super K1,2 -cut of LT Qn such that n2  ≤ |F | ≤ n − 2. Suppose that C is the smallest component of LT Qn − F . • Case 1: |V (C)| = 2. Let V (C) = {x, y}. Because LT Qn has no cycle of order three, |NLT Qn ({x, y})| = 2n − 2. By Lemma 2, |Λ ∩ NLT Qn ({x, y})| ≤ 2 for each Λ ∈ F . Thus, the cardinality of F should be at least |F | ≥ |N ({x,y})|  = 2n−2

LT Qn2 2  = n − 1 > |F |, resulting in a contradiction. • Case 2: |V (C)| ≥ 3. It follows from Lemma 4 that κ2 (LT Qn ) = 3n − 5. However, the total number of nodes in F does not exceed 3|F | ≤ 3(n − 2) = 3n − 6 < 3n − 5 = κ2 (LT Qn ). This implies that C is unlikely to include three or more nodes, resulting in a contradiction. To complete the proof, |F | must be greater than n − 2.

 

Lemmas 7 and 8 implies the following theorem. Theorem 2. For n ≥ 4, κ (LT Qn |K1,2 ) = n − 1. Theorem 3. For n ≥ 5, ⎧ 3n if ⎪ 4 ⎪ ⎨ 3n−3 if  4 κ (LT Qn |K1,3 ) ≤ 3n−2 if ⎪ 4 ⎪ ⎩ 3n−1 if 4

n≡0 n≡1 n≡2 n≡3

(mod (mod (mod (mod

4) 4) 4) 4).

(2)

Proof. Let x and y be any nodes of LT Qn that are 0-neighbors of each other. Without loss of generality, we assume that the rightmost bit of x is zero.

272

Y.-H. Teng and T.-L. Kung

• Case 1: n ≡ 0 (mod 4). For 1 ≤ i ≤

n−2 2 ,

let

Yi = {((x)2i−1 )2i , (x)2i−1 , (x)2i , (y)2i }. By Lemma 1, LT Qn [Yi ] is isomorphic to K1,3 . For 1 ≤ j ≤ n4 , let Tj = {((y)4j−3 )4j−1 , (y)4j−3 , (y)4j−1 , (((y)4j−3 )4j−1 )n−1 }. By Lemma 1, LT Qn [Tj ] is isomorphic to K1,3 . Let S = {(x)n−1 , ((x)n−1 )0 , ((x)n−1 )1 ((x)n−1 )2 }. Obviously, LT Qn [S] is isomorphic to K1,3 .  n−2  n4 2 As NLT Qn ({x, y}) ⊂ i=1 Yi ∪ j=1 Tj ∪ S, F = {Yi |1 ≤ i ≤ n−2 2 } ∪ n {Tj |1 ≤ j ≤ 4 } ∪ {S} is a K1,3 -cut of LT Qn . Because LT Qn [{x, y}] is the smallest component of LT Qn − F , κ (LT Qn |K1,3 ) is bounded above by n 3n |F | = n−2 2 + 4 +1= 4 . • Case 2: n ≡ 1 (mod 4). For 1 ≤ i ≤ n−1 2 , let Yi = {((x)2i−1 )2i , (x)2i−1 , (x)2i , (y)2i }. By Lemma 1, LT Qn [Yi ] is isomorphic to K1,3 . For 1 ≤ j ≤ n−1 4 , let Tj = {((y)4j−3 )4j−1 , (y)4j−3 , (y)4j−1 , (((y)4j−3 )4j−1 )n−1 }. By Lemma 1, LT Qn [Tj ] is isomorphic to K1,3 .  n−1  n−1 2 4 As NLT Qn ({x, y}) ⊂ i=1 Yi ∪ j=1 Tj , F = {Yi |1 ≤ i ≤ n−1 2 }∪{Tj |1 ≤ j ≤ n−1 4 } is a K1,3 -cut of LT Qn . Because LT Qn [{x, y}] is the smallest component n−1 3n−3 of LT Qn − F , κ (LT Qn |K1,3 ) is bounded above by |F | = n−1 2 + 4 = 4 . n−2 • Case 3: n ≡ 2(mod 4). For 1 ≤ i ≤ 2 , let Yi = {((x)2i−1 )2i , (x)2i−1 , (x)2i , (y)2i }. By Lemma 1, LT Qn [Yi ] is isomorphic to K1,3 . For 1 ≤ j ≤ n−2 4 , let Tj = {((y)4j−3 )4j−1 , (y)4j−3 , (y)4j−1 , (((y)4j−3 )4j−1 )n−1 }. By Lemma 1, LT Qn [Tj ] is isomorphic to K1,3 . Let S = {((x)n−1 )n−2 , (x)n−1 , (y)n−1 , (((x)n−1 )n−2 )1 }. By Lemma 1, LT Qn [S] is isomorphic to K1,3 .  n−2  n−2 2 4 As NLT Qn ({x, y}) ⊂ i=1 Yi ∪ j=1 Tj ∪ S, F = {Yi |1 ≤ i ≤ n−2 2 } ∪ n−2 {Tj |1 ≤ j ≤ 4 } ∪ {S} is a K1,3 -cut of LT Qn . Because LT Qn [{x, y}] is the smallest component of LT Qn − F , κ (LT Qn |K1,3 ) is bounded above by n−2 3n−2 |F | = n−2 2 + 4 +1= 4 .

Super K1,p -Connectivity of Locally Twisted Cubes

• Case 4: n ≡ 3(mod 4). For 1 ≤ i ≤

n−1 2 ,

273

let

Yi = {((x)2i−1 )2i , (x)2i−1 , (x)2i , (y)2i }. By Lemma 1, LT Qn [Yi ] is isomorphic to K1,3 . For 1 ≤ j ≤ n−3 4 , let Tj = {((y)4j−3 )4j−1 , (y)4j−3 , (y)4j−1 , (((y)4j−3 )4j−1 )n−1 }. By Lemma 1, LT Qn [Tj ] is isomorphic to K1,3 . Let S = {(y)n−1 , ((y)n−1 )0 , ((y)n−1 )1 ((y)n−1 )2 }. Obviously, LT Qn [S] is isomorphic to K1,3 .  n−1  n−3 2 4 As NLT Qn ({x, y}) ⊂ i=1 Yi ∪ j=1 Tj ∪ {S}, F = {Yi |1 ≤ i ≤ n−1 2 } ∪ n−3 {Tj |1 ≤ j ≤ 4 } ∪ {S} is a K1,3 -cut of LT Qn . Because LT Qn [{x, y}] is the smallest component of LT Qn − F , κ (LT Qn |K1,3 ) is bounded above by n−3 3n−1 |F | = n−1 2 + 4 +1= 4 . The proof is completed.

4

 

Conclusion

The motivation of K1,p -cut, p ≥ 1, arises from the scenario that each node can damage some of its neighborhood simultaneously. The K1,p -connectivity is a reasonable assessment for a network’s connectedness level. In this paper, we determine the super K1,p -connectivity of locally twisted cubes for p = 1, 2, 3. With regard to our future study, a more generalized formula of κ (LT Qn |K1,p ) with p ≤ n will be taken into consideration. Acknowledgements. This work is supported in part by the Ministry of Science and Technology, Taiwan, under Grant No. MOST 109-2221-E-468-009-MY2.

References 1. Akers, S.B., Krishnamurthy, B.: A group theoretic model for symmetric interconnection networks. IEEE Trans. Comput. 38(4), 555–566 (1989) 2. Bondy, J.A., Murty, U.S.R.: Graph Theory. Springer, London (2008) 3. Chang, N.W., Hsieh, S.Y.: {2, 3}-extraconnectivities of hypercube-like networks. J. Comput. Syst. Sci. 79, 669–688 (2013) 4. Dally, W.J., Towles, B.: Principles and Practices of Interconnection Networks. Morgan Kaufmann, San Francisco (2004) 5. Day, K., Tripathi, A.: Arrangement graphs: a class of generalized star graphs. Inf. Process. Lett. 42(5), 235–241 (1992) 6. Efe, K.: The crossed cube architecture for parallel computing. IEEE Trans. Parallel Distrib. Syst. 3, 513–524 (1992) 7. F´ abrega, J., Fiol, M.A.: On the extraconnectivity of graphs. Discret. Math. 155, 49–57 (1996)

274

Y.-H. Teng and T.-L. Kung

8. Flahive, M., Bose, B.: The topology of gaussian and Eisenstein-Jacobi interconnection networks. IEEE Trans. Parallel Distrib. Syst. 21(8), 1132–1142 (2010) 9. Han, Y., Fan, J., Zhang, S., Yang, J., Qian, P.: Embedding meshes into locally twisted cubes. Inf. Sci. 180, 3794–3805 (2010) 10. Harary, F., Hayes, J.P., Wu, H.J.: A survey of the theory of hypercube graphs. Comput. Math. Appl. 15, 277–289 (1988) 11. Hsieh, S.Y., Tu, C.J.: Constructing edge-disjoint spanning trees in locally twisted cubes. Theor. Comput. Sci. 410, 926–932 (2009) 12. Hsieh, S.Y., Wu, C.Y.: Edge-fault-tolerant hamiltonicity of locally twisted cubes under conditional edge faults. J. Comb. Optim. 19, 16–30 (2010) 13. Hsu, L.H., Lin, C.K.: Graph Theory and Interconnection Networks. CRC Press, New York (2008) 14. Hung, R.W.: Embedding two edge-disjoint Hamiltonian cycles into locally twisted cubes. Theor. Comput. Sci. 412, 4747–4753 (2011) 15. Kung, T.L.: Flexible cycle embedding in the locally twisted cube with nodes positioned at any prescribed distance. Inf. Sci. 242, 92–102 (2013) 16. Kung, T.L., Chen, H.C.: Improving the panconnectedness property of locally twisted cubes. Int. J. Comput. Math. 91(9), 1863–1873 (2014) 17. Kung, T.L., Chen, H.C., Lin, C.H., Hsu, L.H.: Three types of two-disjoint-cyclecover pancyclicity and their applications to cycle embedding in locally twisted cubes. Comput. J. 64(1), 27–37 (2021) 18. Kung, T.L., Lin, C.K.: Cluster connectivity of hypercube-based networks under the super fault-tolerance condition. Discret. Appl. Math. 293, 143–156 (2021) 19. Kung, T.L., Teng, Y.H., Lin, C.K.: Super fault-tolerance assessment of locally twisted cubes based on the structure connectivity. Theor. Comput. Sci. 889, 25– 40 (2021) 20. Leighton, F.T.: Introduction to Parallel Algorithms and Architectures: Arrays · Trees · Hypercubes. Morgan Kaufmann, San Mateo (1992) 21. Li, T.K., Lai, C.J., Tsai, C.H.: A novel algorithm to embed a multi-dimensional torus into a locally twisted cube. Theor. Comput. Sci. 412, 2418–2424 (2011) 22. Loh, P.K.K., Hsu, W., Pan, Y.: The exchanged hypercube. IEEE Trans. Parallel Distrib. Syst. 16(9), 866–874 (2005) 23. Ma, M., Xu, J.M.: Panconnectivity of locally twisted cubes. Appl. Math. Lett. 19, 673–677 (2006) 24. Ma, M., Xu, J.M.: Weak edge-pancyclicity of locally twisted cubes. ARS Comb. 89, 89–94 (2008) 25. Mart´ınez, C., Beivide, R., Stafford, E., Moret´ o, M., Gabidulin, E.: Modeling toroidal networks with the gaussian integers. IEEE Trans. Comput. 57(8), 1046– 1056 (2008) 26. Menger, K.: Zur allgemeinen kurventheorie. Fundam. Math. 10, 96–115 (1927) 27. Saad, Y., Schultz, M.H.: Topological properties of hypercubes. IEEE Tran. Comput. 37, 867–872 (1988) 28. Wei, C.C., Hsieh, S.Y.: h-restricted connectivity of locally twisted cubes. Discret. Appl. Math. 217, 330–339 (2017) 29. Xu, X., Zhai, W., Xu, J.M., Deng, A., Yang, Y.: Fault-tolerant edge-pancyclicity of locally twisted cubes. Inf. Sci. 181, 2268–2277 (2011) 30. Yang, X., Evans, D.J., Megson, G.M.: The locally twisted cubes. Int. J. Comput. Math. 82, 401–413 (2005) 31. Yang, X., Megson, G.M., Evans, D.J.: Locally twisted cubes are 4-pancyclic. Appl. Math. Lett. 17, 919–925 (2004)

A Study on Image Transmission Based on Hopping LoRa Ching-Chuan Wei(B) , Kuan-Chun Chang, and Chia-Chi Chang Department of Information and Communication Engineering, Chaoyang University of Technology, Taichung City, Taiwan (R.O.C.) {ccwei,s10930613,ccchang}@cyut.edu.tw

Abstract. LoRa is a low power wide area network (LPWAN) communication that can transmit over long distances under the situation of line of sight, and it is only suitable for transferring small data. Nevertheless, obstacles will severely cause the packet losses and reduce the transmission distance. Hence, we used different spreading factors (SF) to design a hopping network transmission to avoid the obstacle interference and actually achieved the practical long distance communication. In the paper only one-relay for hopping node was tested for feasibility. In addition, to balance the load of the node, every time the picture was transmitted with different cutting ratio. Experimental results show that the one-relay transmission with 1/2 cutting ratio achieves the smallest transmission time. Therefore, this paper verifies the feasibility of hopping image transmission using LoRa.

1 Introduction In the applications of the Internet of Things (IOT), the image transmission of long distance with kilometers have gradually become indispensable in security monitoring, plant growth record, animal monitoring, environmental observations and records, etc. However, the traditional wireless communication technologies such as RFID, Bluetooth, Zigbee, Wi-Fi, etc. having high power consumption and short transmission distance does not meet the requirement of the above applications. Then, the newly proposed low power wide area network (LPWAN) technology like LoRa becomes the potential choice [1–3]. The technology of using LoRa to transmit pictures is just getting started [4, 5]. In the actual deployment the long distance transmission of pictures will always be blocked by tall buildings, trees, mountains, etc. and result into the packet loss or unreceivable condition. Therefore, multi-hop routing is required to solve these problems [6, 7]. The structure of the paper is as follows. In Sect. 2, we present system design. In Sect. 3, we show the experimental results. In Sect. 4 are given conclusions.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 275–282, 2022. https://doi.org/10.1007/978-3-031-08819-3_28

276

C.-C. Wei et al.

2 System Design LoRa parameters including spreading factor (SF), bandwidth (BW), coding rate (CR) and transmission power (TP) have a direct impact on communication performance [8]. SF has a choice between 6 and 12. A larger spreading factor will increase the bit rate. The increasing SF not only increases the transmission distance but also the transmission time. In addition, the signals of different spreading factors will be orthogonal to each other, and that means the signals will not interfere with each other. By means of this characteristic, it had ever been used to improve the LoRa image transmission time [9]. In this article we also used this characteristic to design the hopping node i.e. the relay. The one-relay LoRa image transmission was proposed to solve the obstacle interference and test the feasibility. The architecture consisting of one sender, one relay and one receiver is shown in Fig. 1. The captured image was firstly cut into two half images for load balance. The two half images were then sequently transmitted by the sender, transceived again by the relay 1 node and received by the receiver. Inside the relay 1 node we used SF = 7 to receive the signal from image sender and SF = 8 to transmit the signal to the receiver. Because of the signal orthogonality we thus can simultaneously transmit and receive the signals to reduce the transmission time. The Relay 1 consists of Relay 1–1 and Relay 1–2. Within relay 1 the image data received by the node of SF = 7 (Relay 1–1) will deliver the data to the node of SF = 8 (Relay 1–2) using MQTT (Message Queuing Telemetry Transport), which is a lightweight transport communication protocol designed for the Internet of Things [10].

image

image

Relay1-1

Relay1-2

Fig. 1. One-relay LoRa image transmission architecture

The timing diagram of one-relay LoRa image transmission system is shown in Fig. 2. There are five transmission time slot for one picture transmission. In the first time slot the sender used LoRa to send the required packets to Relay 1–1. In the second time slot the inside communication of the Relay 1 starts. In other words, the Relay 1–1 sent the data to Relay 1–2 through MQTT. In the third time slot the Relay 1–2 transmitted the packets to the Receiver using LoRa, and then completed the task of sending the half image data. At the same time, i.e. the third time slot, the Sender also sent the rest of the half image data to Relay 1–1, and then repeated the above described procedure to finish sending a complete picture. All the transmitting and receiving process adopts ACK (acknowledge) mechanism to ensure the complete transfer of data.

A Study on Image Transmission Based on Hopping LoRa

277

Fig. 2. Timing diagram of one-relay LoRa image transmission system

The flow chart is shown in Fig. 3. At the beginning, initialize the system and then read the test image data of JPEG file. Next, cut the picture into two 1/2 image and continue the LoRa transmission procedure. When the receiver receives the image data packet, it will immediately determine whether there are packet losses. If any packets were lost, their ID numbers will be immediately sent back to LoRa Sender to retransmit until the packets were successfully transmitted. After receiving the last packet, combine the two 1/2 image data into the original image.

3 Experiments and Results The hardware of transmitter, relay and receiver equipment respectively composed of Raspberry pi 3 B+ and Semtech sx1276 LoRa chip. The power part is powered by 5V mobile power supply. The experimental deployment including sender, relay 1 and receiver is shown in Fig. 4. The sender node was placed at Dafeng Bridge, Dali District, Taichung. The relay node was placed at Caohu Bridge, Dali District, Taichung. The receiving node will be placed on the 9th floor of the building of Chaoyang University of Technology. The actual placement of the sender, relay and receiver are shown in Fig. 5. Because there are some obstacles between the sender and receiver, the LoRa communication will be blocked, and the packets were lost. Hence, we looked for the middle place to set the relay 1, where line of sight exists between sender, relay 1 and receiver.

278

C.-C. Wei et al.

Fig. 3. Flow chart of one-relay LoRa image transmission system

In the experiment, the parameters of LoRa are fixed and described in the following, frequency band = 868 MHz, TP (Transmission Power) = 17 dBm, BW (Bandwidth) = 500 kHz, CR (Coding Rate) = 4/5. The test picture shown in Fig. 6 was captured by the Pi camera placed at the sender, and it is simulated for the river monitoring. Its size is 240 × 179 pixels, and JPEG format was used to encode it to 10.2 KB. Then, it was distributed into 85 LoRa packets. Because the receiver is idle in the period when the transmitter transmits the data to the relay, we thus attempt to distribute the load to every node From the viewpoint of

A Study on Image Transmission Based on Hopping LoRa

Relay 1 Sender

Receiver

Fig. 4. Experimental deployment including sender, relay 1 and receiver

Fig. 5. Actual placement of the sender, relay and receiver in (a), (b) and (c)

Fig. 6. Test picture captured by the Pi camera placed at the sender

279

280

C.-C. Wei et al.

time. In order to approach this goal, we try to cut the picture into smaller size to make the receiver receive the image data earlier. To test the performance of the cutting ratio of the picture, we evaluate the transmission time of various cutting ratio including 1(no cutting), 1/2, 1/3 and 1/4 for comparison. The cutting ratio of 1/2 means that the image sensor node and relay will completely transmit 1/2 of the picture every time. The similar situation applies to the cutting ratio of 1/3 and 1/4. The pictures of different cutting ratios are shown in Fig. 7. PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity Index) are used to measure the similarity between the image before transmission and that after transmission. Both of them have high values in this experiment. That means the low distortion of the transmitted image. Therefore, we mainly discussed the influence on transmission time in the experiment. The experimental results are shown in Fig. 8. The fastest transmission time (=69 s) occurred in cutting ratio = 1/2 of the one relay transmission. We can thus know that resulting from the optimum load balance the one-half cutting ratio approaches the best performance of transmission time for onerelay LoRa image transmission system. The transmission time of one-half cutting ratio decreases almost about twenty seconds compared with that of no cutting (cutting ratio = 1).

cutting ratio = 1(no cutting)

cutting ratio = 1/3

cutting ratio = 1/2

cutting ratio = 1/4

Fig. 7. Pictures of different cutting ratios

281

Transmission time(s)

A Study on Image Transmission Based on Hopping LoRa

Cutting ratio Fig. 8. Transmission time for various cutting ratio of the test picture

4 Conclusion There are two contributions in this paper. The first point is that we choose the transmission method of multi-hop network in an open place to bypass the building obstacle. The onerelay hopping transmission system was successfully tested to verify the feasibility of this method. The second point is that we don’t make the nodes to waste too much waiting time to receive the data from the previous one. In other words, we divide the original picture into several parts. Every time the node only transmits a single part so that the node will not waste too much time to receive or transmit the data. Therefore, the transmission time can be decreased.

References 1. Huh, H., Kim, J.Y.: LoRa-based mesh network for IoT applications. In: IEEE 5th World Forum on Internet of Things (WF-IoT) (2019) 2. Rawat, A.S., et al.: LoRa (long range) and LoRaWAN technology for IoT applications in COVID-19 pandemic. In: IEEE International Conference on Advances in Computing, Communication and Materials (ICACCM) (2020) 3. Augustin, A., Yi, J., Clausen, T., Townsley, W.M.: A study of LoRa: long range & low power networks for the Internet of Things. Sensors 16(9), 1466 (2016) 4. Jebril, A.H., Sali, A., Ismail A., Rasid, M.F.A.: Overcoming limitations of LoRa physical layer in image transmission. Sensors (Basel) 18(10) (2018) 5. Chen, T., Eager D., Makaroff, D.: Efficient image transmission using LoRa technology in agricultural monitoring IoT systems. In: International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData), pp. 937–944 (2019) 6. Azhari, R.Y., Firmansyah, E., Bejo, A.: Simple protocol design of multi-hop network in Lora. In: IEEE International Seminar on Research of Information Technology and Intelligent Systems (2019)

282

C.-C. Wei et al.

7. Branch, P., Cricenti, T.: A LoRa based wireless relay network for actuator data. In: IEEE International Conference on Information Networking (ICOIN) (2020) 8. Alsaid, A., et al.: A new approach towards LoRa wireless technology parameters’ selection. In: IEEE Jordan International Joint Conference on Electrical Engineering and Information Technology (JEEIT) (2021) 9. Wei, C.C., Chen, S.T., Su, P.Y.: Image transmission using LoRa technology with various spreading factors. In: 2nd World Symposium on Communication Engineering (WSCE) (2019) 10. mqtt.org. http://mqtt.org

A Conditional Local Diagnosis Algorithm on the Arrangement Graph Tzu-Liang Kung1 , Cheng-Kuan Lin2 , and Yuan-Hsiang Teng3(B) 1

2

Department of Computer Science and Information Engineering, Asia University, Taichung City 413, Taiwan, R.O.C. [email protected] Department of Computer Science, National Yang Ming Chiao Tung University, Hsinchu 30010, Taiwan, R.O.C. [email protected] 3 Department of Computer Science and Information Engineering, Providence University, Taichung City 433, Taiwan, R.O.C. [email protected]

Abstract. In this article, we design a conditional local diagnosis algorithm for an arrangement graph. Suppose that u is a vertex in an arrangement graph An,k . If F is a conditional faulty set in An,k with |F | ≤ k(n − k), then the status of u can be identified accurately with our algorithm under the BGM diagnosis model.

1

Introduction

Sensor networks have gained more and more popularity in computer technology. In a sensor network, there are more than one sensor node or processor. These sensor nodes, processors and the links of a system are usually modeled as a graph topology. The reliability of a system is crucial since even a few malfunctions would disable to operate the service. Whenever devices are found to be faulty, they should be replaced with fault-free ones as soon as possible to guarantee that the system can work properly. Thus the ability of identifying all the faulty devices in a system is very important. This is known as system diagnosis. The diagnosability is the maximum number of faulty devices that can be identified correctly. A system is t-diagnosable if all the faulty devices can be pointed out precisely with the faulty devices being t at most. Many results about the diagnosis and the diagnosability have been proposed in literature [6,7,9,11,12,14,15]. Barsi, Grandoni, and Maestrini have proposed the BGM diagnosis model. The BGM model is the tested-based diagnosis. Under the BGM model, a processor performs the diagnosis by testing the neighboring processors via the links between them. Some related studies have appeared in the literature [1,2,8,13]. The arrangement graph [3] was proposed by Day and Tripathi as a generalization of the star graph. It is more flexible in its size than the star graph. Given two positive integers n and k with n > k, the arrangement graph An,k is the graph (V, E), where V = {p | p is an arrangement of k elements out of the c The Author(s), under exclusive license to Springer Nature Switzerland AG 2022  L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 283–289, 2022. https://doi.org/10.1007/978-3-031-08819-3_29

284

T.-L. Kung et al.

n symbols: 1, 2, ..., n} and E = {(p, q) | p, q ∈ V and p, q differ in exactly one position}. The arrangement graph An,k is a regular graph of degree k(n−k) with n! 3k (n−k)! vertices. The diameter of An,k is  2 . An,1 is isomorphic to the complete graph Kn and An,n−1 is isomorphic to the n-dimensional star graph. Moreover, An,k is vertex symmetric and edge symmetric [3]. Many related works about the arrangement graph have appeared in the literature [3,4,10]. In this paper, we design a conditional local diagnosis algorithm for an arrangement graph. Suppose that u is a vertex in an arrangement graph An,k . If F is a conditional faulty set in An,k with |F | ≤ k(n − k), then the status of u can be identified accurately with our algorithm under the BGM diagnosis model.

2

Preliminaries

For the graph definitions and notations, we follow [5]. Let G = (V, E) be a graph if V is a finite set and E is a subset of {{u, v} | {u, v} is an unordered pair of V }. We say that V is the vertex set and E is the edge set of G. Two vertices u and v are adjacent if {u, v} ∈ E; we say u is a neighbor of v, and vice versa. We use NG (u) to denote the neighborhood set {v | {u, v} ∈ E(G)}. The degree of a vertex v in a graph G, denoted by degG (v), is the number of edges incident to v. Under the BGM diagnosis model, we assume that adjacent processors are capable of performing tests on each other. Let G = (V, E) denote the underlying topology of a multiprocessor system. For any two adjacent vertices u, v ∈ V (G), the ordered pair (u, v) represents a test that processor u is able to diagnose processor v. In this situation, u is a tester, and v is a testee. Suppose that u is fault-free. The outcome of a test (u, v) is 1 (respectively, 0) if u evaluates v to be faulty (respectively, fault-free). Since the faults considered here are permanent, the outcome of a test is reliable if and only if the tester is fault-free. If u is faulty and v is fault-free, both results of are possible. If u and v are faulty, the result

Fig. 1. The arrangement graph A4,2 .

A Conditional Local Diagnosis Algorithm on the Arrangement Graph

285

of the test (u, v) is 1. See Table 1. A test assignment for system G is a collection of tests and can be modeled as a directed graph T = (V, L), where (u, v) ∈ L implies that u and v are adjacent in G. The collection of all test results from the test assignment T is called a syndrome. Formally, a syndrome of T is a mapping σ : L → {0, 1}. A faulty set F is the set of all faulty processors in G. It is noticed that F can be any subset of V . The process of identifying all faulty vertices is said to be the system diagnosis. Furthermore, the maximum number of faulty vertices that can be correctly identified in a system G is called the diagnosability of G, denoted by τ (G). We say a system G is t-diagnosable if all faulty vertices in G can be precisely pointed out with the total number of faulty vertices being at most t. Table 1. Outcomes of (x, y) under the BGM diagnosis model. Vertex x

Vertex y

Outcome of (x, y)

Fault-free Fault-free 0 Fault-free Faulty

1

Faulty

Fault-free 0 or 1

Faulty

Faulty

1

Suppose that G = (V, E). A set F ⊂ V (G) is a conditional faulty set if NG (v) is not a subset of F for any vertex v ∈ V (G) − F . A graph G is conditional faulty if the faulty vertex set of G forms a conditional faulty set. For any two distinct conditional faulty sets F1 and F2 of G with |F1 | ≤ t and |F2 | ≤ t, if (F1 , F2 ) is a distinguishable pair, then G is conditional t-diagnosable. The maximum number of conditional faulty vertices that can be correctly identified in G is called the conditional diagnosability of G, denoted by τ c (G). Let v be any vertex in G. The c (v), is defined to be the conditional local diagnosability of v in G, denoted by τG ∗ maximum integer of t such that G is conditionally t -diagnosable at vertex v. It c (v) | v ∈ V (G)}. is trivial that τ c (G) ≥ τ (G) and τ c (G) = min{τG Let n and k be two positive integers with n > k. And, let n and k

denote the sets {1, 2, ..., n} and {1, 2, ..., k}, respectively. Then, the vertex set of the arrangement graph An,k , V (An,k ) = {p | p = p1 p2 ...pk with pi ∈ n for 1 ≤ i ≤ k and pi = pj if i = j} and the edge set of An,k , E(An,k ) = {(p, q) | p, q ∈ V (An,k ), p and q differ in exactly one position }. Figure 1 illustrates A4,2 .

3

Main Result

Two distinct faulty sets F1 and F2 of V are distinguishable if σ(F1 ) ∩ σ(F2 ) = ∅; otherwise, F1 and F2 are indistinguishable. That is, (F1 , F2 ) is a distinguishable pair of faulty sets if σ(F1 ) ∩ σ(F2 ) = ∅. Otherwise, (F1 , F2 ) is an indistinguishable pair. For any two distinct faulty sets F1 and F2 of G with |F1 | ≤ t and |F2 | ≤ t, a graph G is t-diagnosable if and only if (F1 , F2 ) is a distinguishable pair. Let F1

286

T.-L. Kung et al.

u

u

F1

v

F2

F1

u F1

v F2

u F2

F1

v

F2 v

Fig. 2. An illustration of Theorem 1.

and F2 be two distinct sets. We use F1  F2 to denote the symmetric difference (F1 − F2 ) ∪ (F2 − F1 ) between F1 and F2 . Lin et al. presented a sufficient and necessary characterization of t-diagnosable systems under the BGM model in [8]. See Fig. 2 for an illustration. Theorem 1. [8] Let F1 and F2 be any two distinct vertices subsets of a system G = (V, E). Thus, (F1 , F2 ) is a distinguishable pair if and only if one of the following states holds: 1. There exist a vertex u ∈ V − (F1 ∪ F2 ) and a vertex v ∈ F1  F2 such that (u, v) ∈ E; 2. There exist two distinct vertices u, v ∈ F1 − F2 such that (u, v) ∈ E; 3. There exist two distinct vertices u, v ∈ F2 − F1 such that (u, v) ∈ E. Theorem 2. [8] Let G = (V, E) be a system, and u ∈ V (G). G is locally tdiagnosable at vertex u if and only if for any two distinct sets F1 , F2 ⊂ V with |F1 | ≤ t, |F2 | ≤ t and u ∈ F1  F2 , (F1 , F2 ) is a distinguishable pair. Suppose that k ≥ 1 and n − k ≥ 2. A set F ⊂ V (An,k ) is a conditional faulty set if NAn,k (v) is not a subset of F for any vertex v ∈ V (An,k ) − F in an arrangement graph An,k . An arrangement graph An,k is conditionally faulty if the faulty vertex set of An,k forms a conditional faulty set. For any two distinct conditional faulty sets F1 and F2 of An,k with |F1 | ≤ k(n−k) and |F2 | ≤ k(n−k), if (F1 , F2 ) is a distinguishable pair, An,k is conditionally k(n − k)-diagnosable. Now, we prove our main result. Theorem 3. Suppose that k ≥ 1 and n − k ≥ 2. Let F1 and F2 be two distinct conditional faulty sets in an arrangement graph An,k with |F1 | ≤ k(n − k) and |F2 | ≤ k(n−k). Thus, An,k is conditionally k(n−k)-diagnosable under the BGM diagnosis model.

A Conditional Local Diagnosis Algorithm on the Arrangement Graph

287

Proof. Without loss of generality, we consider the faulty set F1 . Let u be a vertex in V (An,k ) − F1 . Because F1 is a conditional faulty set, a vertex v is adjacent to u such that v ∈ / F1 . By Theorems 1 and 2, An,k is conditionally k(n − k)diagnosable under the BGM diagnosis model. Now, we give the conditional local diagnosis algorithm for determining the fault status of a vertex u in an arrangement graph An,k under the BGM model in Algorithm 1. Assume that k ≥ 1 and n − k ≥ 2. We now prove that a vertex u in an arrangement graph An,k can be diagnosed accurately with Algorithm 1 under the BGM diagnosis model.

Algorithm 1: DAA(n, k, u) Input: An arrangement graph An,k with n − k ≥ 2 and a vertex u ∈ V (An,k ). Output: The value is 0 or 1 if u is fault-free or faulty, respectively. begin Perform the tests (u, vi ) and (vi , u), where vi ∈ NAn,k (u) for every 1 ≤ i ≤ k(n − k). if there exists at least one test result pair ((u, vi ), (vi , u)) = (0, 0) then return 0; end else return 1; end end

Theorem 4. Let k ≥ 1 and n − k ≥ 2. Suppose that u is a vertex in an arrangement graph An,k . If F is a conditional faulty set in An,k with |F | ≤ k(n−k), then the faulty/fault-free status of u can be identified accurately with DAA(n, k, u) under the BGM diagnosis model. Proof. Depending on the definition of the BGM model and the results listed in Table 1, if u ∈ F , test result pair ((u, vi ), (vi , u)) ∈ {(0, 1), (1, 0), (1, 1)}, where / F . According to vi ∈ NAn,k (u) for every 1 ≤ i ≤ k(n − k). Suppose that u ∈ the definition of a conditionally faulty graph, there exists at least one vertex / F . Thus, we have the vi ∈ NAn,k (u) for some 1 ≤ i ≤ k(n − k) such that vi ∈ test result pair ((u, vi ), (vi , u)) = (0, 0) and the theorem holds.

4

Conclusion

We design a conditional local diagnosis algorithm for an arrangement graph in this article, and prove that the faulty/fault-free status of a vertex in an arrangement graph can be identified accurately with our algorithm under the

288

T.-L. Kung et al.

BGM diagnosis model. Teng and Lin [11] propose the concept about test round controllable adaptive local diagnosis algorithm. They design an adaptive local diagnosis algorithm for a network in k test rounds. In [14], Wang et al. propose the three test rounds local diagnosis algorithm for the 2D-tours network. Lin et al. prove that the vertex in an n-dimensional hypercube-like network can be diagnosed correctly with a faulty set F in three test rounds under the BGM diagnosis model if |F | ≤ n in [8]. Future research will develop the test round controllable adaptive local diagnosis algorithm for the arrangement graph under the BGM diagnosis model. Acknowledgements. This work was supported in part by the Ministry of Science and Technology of the Republic of China under Contract MOST 109-2221-E-468-009-MY2.

References 1. Albini, L.C.P., Chessa, S., Maestrini, P.: Diagnosis of symmetric graphs under the BGM model. Comput. J. 47, 85–92 (2004) 2. Blough, D.M., Brown, H.W.: The broadcast comparison model for on-line fault diagnosis in multicomputer systems: theory and implementation. IEEE Trans. Comput. 48, 470–493 (1999) 3. Day, K., Tripathi, A.: Arrangement graphs: a class of generalized star graphs. Inf. Process. Lett. 42(5), 235–241 (1992) 4. Hsu, H.C., Li, T.K., Tan, J.M., Hsu, L.H.: Fault Hamiltonicity and fault Hamiltonian connectivity of the arrangement graphs. IEEE Trans. Comput. 53(1), 39–53 (2004) 5. Hsu, L.H., Lin, C.K.: Graph Theory and Interconnection Networks, CRC Press, Boca Raton (2008) 6. Kung, T.L., Teng, Y.H., Lin, C.K., Chen, H.C.: A localized fault detection algorithm for mobility management in the strongly t-diagnosable wireless ad hoc network under the comparison model. EURASIP J. Wirel. Commun. Netw. 2016(1), 218 (2016) 7. Li, X., Teng, Y.H., Kung, T.L., Chen, Q., Lin, C.K.: The diagnosability and 1-goodneighbor conditional diagnosability of hypercubes with missing links and brokendown nodes. Inf. Process. Lett. 146, 20–26 (2019) 8. Lin, C.K., Kung, T.L., Hung, C.N., Teng, Y.H.: A local diagnosis algorithm for hypercube-like networks under the BGM diagnosis model. Fundamenta Informaticae (2022) 9. Lin, C.K., Kung, T.L., Wang, D., Teng, Y.H.: The diagnosability of (K4 −{e})-free graphs under the PMC diagnosis model. Fund. Inform. 177(2), 181–188 (2020) 10. Teng, Y.H.: The spanning connectivity of the arrangement graphs. J. Parallel Distrib. Comput. 98, 1–7 (2016) 11. Teng, Y.H., Lin, C.K.: A test round controllable local diagnosis algorithm under the PMC diagnosis model. Appl. Math. Comput. 244, 613–623 (2014) 12. Tsai, C.H.: A quick pessimistic diagnosis algorithm for hypercube-like multiprocessor systems under the PMC model. IEEE Trans. Comput. 62(2), 259–267 (2013) 13. Vedeshenkov, V.A.: On the BGM model-based diagnosis of failed modules and communication lines in digital systems. Autom. Remote. Control. 63, 316–327 (2002)

A Conditional Local Diagnosis Algorithm on the Arrangement Graph

289

14. Wang, L., Liu, N., Lin, C.K., Kung, T.L., Teng, Y.H.: A diagnosis algorithm on the 2D-torus network. In: The 12th International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing (IMIS-2018), Matsue, Japan (2018) 15. Zhou, S., Wang, J., Xu, X., Xu, J.M.: Conditional fault diagnosis of bubble sort graphs under the PMC model. Intell. Comput. Evol. Comput. 180, 53–59 (2013)

Application of Artificial Intelligence Technology in the Design of Hand Training and Intelligence Training for Patients with Dementia Wei-Chun Hsu1,2 and Hsing-Chung Chen2,3(B) 1 Department of Electrical Engineering, WuFeng University, 117, Sec 2, Chiankuo Road,

Minhsiung 62153, Chiayi County, Taiwan, R.O.C. [email protected] 2 Computer Science and Information Engineering, Asia University, 500, Lioufeng Road, Wufeng, Taichung 413305, Taiwan, R.O.C. [email protected], [email protected] 3 Department of Medical Research, China Medical University Hospital, China Medical University, Taichung 110122, Taiwan

Abstract. This study proposed a training program combining hand movement ability, hearing ability, cognition, and intelligence with the help of artificial intelligence technology. This training program needs a projector with high lumen and a wall with a whiteboard or a wall painted with magnetic paint. The design content generated by artificial intelligence technology can be projected to the wall and show the training procedures. The patients with dementia will follow the training program and train their hands and body movements and the cognitive function of the brain by holding the magnetic patch in their hand and sticking it in the synthetic position on the screen. The computer can play background music by the design of the activity content to perform music therapy during the training process. Through group operation, the training program will improve or maintain the patients’ current hand and limb movement ability and cognitive function. And through group activities to cultivate the tacit understanding of group cooperation, as well as maintain interpersonal interaction, and slow down aging.

1 Introduction Due to the advancement of medical technology, the life span of human beings has been prolonged. Because of the problems of modernization, various factors that are not conducive to physical and mental health have also been derived. In today’s society, there is a problem of dementia. To delay the deterioration of the dementia problem, the corresponding actions need to be taken to face this problem. Appropriate physical activity can achieve the benefits of preventing dementia. There are researches [1, 2] that indicated that the risk of dementia is affected by gender, genotype, exercise pattern, and social factors. Through appropriate activities, aerobic exercise, increasing the frequency of exercise, and community group health activities for the elderly, the blood flow to the brain can be increased to improve the cognitive function of the brain. Dementia can be controlled after drug treatment, but the course of the disease will worsen after some © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 290–296, 2022. https://doi.org/10.1007/978-3-031-08819-3_30

Application of Artificial Intelligence Technology

291

time. To effectively control dementia, the treatments [3] can be combined with cognitive therapy, exercise, mental and behavioral rehabilitation, etc. Dementia affects people in physical, psychological, and social aspects, it becomes a very important goal to reduce the risk of dementia and achieve healthy aging and quality of life in society. [4]. Several clinical studies have also shown that, in addition to currently available drug treatments, cognitive rehabilitation therapy can stabilize or improve mild to moderate dementia. Software and other activities are integrated into standardized tasks to improve cognitive function (such as memory, problem-solving ability, and concentration), and then change the patient’s skills or behavior. People with dementia can perform as functional activities as they can [5]. Cognitive rehabilitation training usually involves memory, attention, language, or executive function. The main purpose of the training is to help patients maintain optimal physical, mental and social function and reduce barriers due to illness or injury. Cognitive rehabilitation is a personalized treatment that must consider the patient’s life experience and social background. For patients with dementia, maintaining their original daily life environment for interventional therapy can help patients make full use of their remaining memory capacity and reduce memory decline. Studies [6, 7] have shown that the effectiveness of training can be improved by the methods of interventional cognitive stimulation including attention training, computer visual-spatial exercises, and repetitive memory training. Due to the advancement of technology, artificial intelligence (AI) technology has also been applied to assist rehabilitators or caregivers to increase the effectiveness of activity training. According to the recording of the training process, AI technology can be used to judge the effect as a future reference. AI technology is also used in the treatment of children’s development. Occupational therapists often judge children’s development status based on the quality of children’s activities, but most of the judgments are subjective and qualitative determination, which lacks the support of objective quantitative data. AI technology can be applied to analyze pictures of children’s activities, and the features of the photos are extracted through computer vision, and these features are further speculated on the development performance [8]. Therefore, AI technology can also be used to analyze the photos of dementia patients’ training activities to further predict campaign effectiveness by the ways of these computer vision capture technology. Since 2002, Kaohsiung Veterans General Hospital has promoted smoking cessation treatment, smoke-free hospitals, and community-based tobacco harm prevention. In recent years, the application of big data and the design of AI interactive programs have significantly improved smokers’ willingness to quit smoking and make it easier for smokers to quit smoking. Smokers increase their willingness to quit smoking in just a few minutes and understand their smoking risks and their chances of quitting successfully! [9] The interactive activity design combined with AI and artificial intelligence [10] could also improve the willingness of dementia patients to participate in training activities to promote the effectiveness of the activities.

292

W.-C. Hsu and H.-C. Chen

2 Group Activity Training of Hand and Intelligence This training program is a combination of hand movement ability, hearing ability, cognition, and intelligence training program. It needs a projector with high lumen and a wall with a whiteboard or a wall painted with magnetic paint, and a laptop is used to project the design content on the magnetic wall. AI-aided computer programs are performed to train hand and body movements and brain cognitive functions by holding magnetic patches in the hands and sticking them on the screen. In the meantime, background music is played to perform music therapy. It is designed to operate in small groups. The time of each activity is set to 40 min, and the time can be increased or decreased according to the actual situation on site. The participants are separated into several groups, which include the caregiver and the care recipient. The number of groups is set to be 6 with 12 participants. Each time 3 groups are asked to play on stage. A space with a highlumen projector is required approximately 6 m long × 6 m wide × 3 m high. The wall with a whiteboard or painted with magnetic paint is shown in Fig. 1, and the activity space planning is shown in Fig. 2. There are three types of activities described in the followings. 2.1 Group Work Puzzle The selected the appropriate design content is projected on the magnetic wall with the help of AI technology. And appropriate background music selected by AI technology is played. The caregiver assists the care recipient to make a magnetic patch collage. In addition to training the hand movement of the care recipient, the hand tactile training is carried out with the collage of different tactile sensations, which also cultivates the spirit of teamwork, and cooperates with the aesthetic perception to complete a work. Then each team member will share their feelings to achieve the purpose of emotional relief. The beginning, simple relaxation activities will be carried out to achieve limb stretching and reduce the damage caused by the activities. The simple balance activities mainly focus on the stretching of the hands. In the process, AI technology is used to play light rhythm music, so that members can enjoy stable and coordinated balance. AI technology can be applied to analyze pictures of patients’ activities, and the features of the photos are extracted through computer vision. The characteristics of the photos can be captured to predict the effectiveness of the activities. 2.2 Group Math Training The projector projects some basic operations of a mathematical addition, subtraction, multiplication, and division with the help of AI technology. Then the teammates take the digital magnetic patch to answer. 10 questions are performed for 5 min, and then the period is gradually reduced to achieve the effect of training brain operations. AI technology is used to analyze the photos of dementia patients’ training activities, and to further predict the effectiveness of the activities.

Application of Artificial Intelligence Technology

293

2.3 Nostalgic Music Training With artificial intelligence technology, the old-time singer’s photo is selected and projected on the wall. And the computer plays the singer’s songs at the same time. Teammates will get credits by answering the right questions about the song and singing the song corrected. And then another team has to sing and answer. The team wins the game with more credits. The game will achieve the benefits of training brain memory. AI technology is used to analyze the photos of dementia patients’ training activities, and to further predict the effectiveness of the activities.

Fig. 1. The wall with a whiteboard or painted with magnetic paint.

Fig. 2. The activity space planning.

The props required for the activity include a projector with high lumen, a wall with a whiteboard or a wall painted with magnetic paint, and a laptop. Magnetic patches, and magnetic objects, as shown in Fig. 3.

294

W.-C. Hsu and H.-C. Chen

Fig. 3. Several magnetic patches.

3 Discussions If the participant has Alzheimer’s disease, the early symptoms are mainly memory loss. The caregiver can help the participant to strengthen the memory part during activities. If the participant has frontotemporal dementia, it is characterized by the early appearance of personality changes and loss of behavioral control, which will lead to unreasonable behaviors. Therefore, caregivers should pay attention to the participants’ behaviors during activities to prevent disordered behaviors. If there is a language barrier problem in the early stage, during the nostalgic music training, the participants can be driven to sing for speech training. Participants with dementia with Lewy bodies may experience stiffness, tremors, gait disturbance, and falls in the early stage in addition to cognitive impairment. Therefore, caregivers should pay attention to the participants’ physical movements during activities to prevent fall, and participants will have more obvious mental symptoms such as visual or auditory hallucinations, emotional instability, or paranoia. Caregivers should pay attention to the behavior of participants during activities to prevent disorderly behavior. If the participant has Parkinson’s disease, the caregiver should pay attention to the participant’s body movements and behavior during the activity. In this program, a small group health activity of 12 people is used to increase the blood flow of the brain to improve the cognitive function of the brain [2]. And cognitive rehabilitation can improve cognitive function (such as memory, problem solving, and concentration). The group work puzzle activity uses group cooperation to complete a wall collage work with different magnetic collages. Participants can directly touch the collages of different touches with their hands and use different words to share their feelings. The content of the works can also allow participants to have the problemsolving ability and concentration [3], and to relax the body and mind by expressing their feelings.

4 Conclusions At present, most of the clinical scales including MMSE, APAS-cog, QoL-AD, and NPI are used to evaluate the cognition of the elderly. MMSE (Mini-Mental State Examination) short intelligence test is a scale proposed by Folstein et al. in 1975 to briefly

Application of Artificial Intelligence Technology

295

test intelligence. There are eleven questions. Cognitive functions such as memory and short-term memory, language ability, and visuospatial ability are assessed in 5 to 10 min. The maximum score is 30 points. The higher the score, the better the ability. A score greater than or equal to 25 points represents normal intelligence. It represents mild with 21–24 points and moderates with 10–20 points. It represents severe intelligence when the score is less than 9 points. ADAS-Cog (Alzheimer’s Disease Assessment Scale-Cognitive) Alzheimer’s Disease Scale, is a special scale for evaluating Alzheimer’s patients and it has been used as a clinical reference for measuring cognitive function. Tools and scales are divided into two parts: subjective and objective. The subjective part mainly evaluates the patients’ oral comprehension, expression, and recall of test guidance; the objective part focuses on orientation, language, graphic construction, concept application, and reading. The test time is about 20–30 min, and the total score is 70 points. The higher the score, the more severe the cognitive impairment. QOL-AD (Quality of Life-Alzheimer’s Disease Scale) Alzheimer’s Quality of Life Scale is a scale proposed by Logsdon et al. in 1999 to measure the clinical quality of life of Alzheimer’s patients. The content of the scale is defined by five aspects: interpersonal relationship, environmental, functional, physiological, and psychological. It covers a healthy body, spirit, emotion, living conditions, memory, family, marriage, friends, self, ability to handle housework, interests, money, and life. Each item is scored 1 to 4 points. The score is 1 point when the subjective feeling is not good and 4 points for the best. It can be used to measure the psychological state of patients and caregivers. For patients with early dementia, appropriate life care and environmental support can slow down early dementia symptoms including memory loss and weakening of dates and times. Cognitive Rehabilitation (CR) is a personalized approach to help dementia patients and their families, and medical staff determine patient sensitivity and confirm residual memory capacity. Cognitive rehabilitation activities can give the brain appropriate stimulation through cognitive activities, and then establish new “connections” between neurons to achieve the purpose of strengthening neuroplasticity and increasing reserve function. The focus of cognitive rehabilitation is to reduce dysfunction and enhance the activities of daily living for maximum social participation and interaction. Cognitive rehabilitation focuses on reducing functional impairment and improving activities of daily living for maximum social participation and interaction. Training activity outcomes were assessed by using the MMSE Short Intelligence Test, ADAS-Cog Alzheimer’s Disease Scale, and QOL-AD Alzheimer’s Quality of Life Scale. The assessments were conducted every 3 months to check whether there is a significant difference after the intervention of health activities and to understand whether cognitive rehabilitation can effectively improve the symptoms of dementia. Acknowledgments. This work was supported by the Ministry of Science and Technology (MOST), Taiwan, under MOST grant numbers below: 111-2218-E-468-001-MBK, 110-2218E-468-001-MBK, 110-2221-E-468-007, and 110-2218-E-002-044. This work was also supported in part by the Ministry of Education under grant number. I109MD040. This work was supported in part by Asia University, Taiwan, and China Medical University Hospital, China Medical University, Taiwan, under grant numbers below: ASIA-110-CMUH-22, ASIA108-CMUH-05, ASIA-106-CMUH-04, and ASIA-105-CMUH-04.

296

W.-C. Hsu and H.-C. Chen

References 1. Lai, Y., Wu, J.: A preliminary study on the integrated analysis of the effect of cognitive rehabilitation on dementia patients. J. Healthy Living Successful Aging 8(1), 29–39 (2016) 2. Wang, J., Cai, J.: Dementia prevention with exercise and physical activity: a review of the literature. Taiwan J. Publ. Health 28(4), 268–277 (2009) 3. Lin, K., Wang, P., Li, X., Chen, R.: Dementia ethics issues: common issues for patients, families and healthcare professionals. Appl. Psychol. Res. 55, 101–113 (2012) 4. Wu, M., Lin, G., Tang, P.: Integrative analysis of Taijiquan on promoting cognitive function of the elderly. Physiotherapy 37(4), 347–358 (2012) 5. Lee, S.S., Powell, N.J., Esdaile, S.: A functional model of cognitive rehabilitation in occupational therapy. Can. J. Occup. Ther. 68(1), 41–50 (2001) 6. Clare, L., Woods, R.T.: Cognitive training and cognitive rehabilitation for people with earlystage Alzheimer’s disease: a review. Neuropsychol. Rehabil. 14(4), 385–401 (2004) 7. Sitzer, D.I., Twamley, E.W., Jeste, D.: Cognitive training in Alzheimer’s disease: a metaanalysis of the literature. Acta Psychiatr. Scand. 114(2), 75–90 (2006) 8. You, Y., Lin, G., Huang, Q., Xie, Q.: Using deep learning to analyze origami photos to estimate the age of children aged 3–6. J. Soc. Occup. Therapy 39(2), 211–231 (2021) 9. Xue, G., Chen, H., Zheng, S., Yu, S.: Application of AI artificial intelligence programs and informatization in smoke-free hospitals. Health Promot. Res. Pract. 4(1), 88–95 (2021) 10. Allahyari, E., Moshtagh, M.: Predicting mental health of prisoners by artificial neural network. BioMedicine 11(1), 3 (2021). https://doi.org/10.37796/2211-8039.1031

An Efficient Disaster Recovery Mechanism for Multi-region Apache Kafka Clusters Lung-Pin Chen, Leu-Fang Yei(B) , and Ying-Ru Chen Department of Computer Science, Tunghai University, Taichung City, Taiwan {lbchen,leufy,g1000000}@thu.edu.tw

Abstract. Today many internet messaging platforms are built based on publishsubscribe event delivery models that needs to deliver large amounts of messages to ubiquitous clients in a timely manner. To deal with the huge amount of messages reliably, fault tolerance and disaster recovery, e.g. power outages, network interruptions, and software failures, have become vital issues. This paper studies the issue of disaster recovery for the popular Apache Kafka message delivery system. We adopt the strategy of cooperative redundant among multi-region Kafka clusters. By using this approach, the internet messaging applications connect to a local Kafka consumer gateway which uses multi-threading to connect multiple Kafka clusters in different regions. The gateway automatically selects active cluster thereby achieves location transparency and fault tolerance.

1 Introduction Kafka is an open-source stream processing platform [1]. The goal of the project is to provide a unified, high-throughput, low-latency platform for processing real-time data. Its persistence layer is a large-scale message queue based on distributed log file transactions. It provides a publish-subscribe event delivery models for modern Internet applications that requires to deliver the massive amounts of messages to ubiquitous clients in a timely manner. This paper addresses the issue of disaster recovery for Kafka clusters. When a Kafka cluster encounters unforeseen disasters such as power outages, network interruptions and software failures, it may lead to data loss during transmission, failure to update in time, and even cause the service suspension so that it is necessary to restore the functionalities as soon as possible, so that the client user is not affected, and at the same time, it can ensure that the user’s information will not be irreparable due to errors. This paper adopts the approach of redundancy among multi-region Kafka clusters. Unlike previous complex approaches [4–6] this work develops a gateway server which hides the complex steps of implementing the redundancy techniques. In this mechanism, an internet messaging application simply connect to the consumer gateway server which uses multi-threading to connect multiple Kafka clusters in different regions. This gateway automatically makes data redundant and selects most approximated server that can provide the messaging service. The gateway server hides the intricate details behind thereby achieves location transparency and fault tolerance. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 297–306, 2022. https://doi.org/10.1007/978-3-031-08819-3_31

298

L.-P. Chen et al.

The rest of this paper is organized as follows. In Sect. 2, we will introduce the Kafka platform. Section 3 will introduce the system architecture and the main algorithms. Section 4 is the experiments section. Finally, concluding remarks and future work are given in Sect. 5.

2 Background This section will explain the architecture of Apache Kafka. It is a high-throughput, distributed subscription-based distributed messaging system. It is widely used for features of high throughput, low latency, scalability, reliability, and asynchronous communication.

Fig. 1. Kafka broker: topic, producer and consumer [1].

2.1 Apache Kafka A Kafka broker is illustrated in Fig. 1. The main roles include topic, producer and consumer, and their related terms are described as follows. • Producer/Consumer A producer (or consumer) is a process that publishes (or reads) messages to (or from) a Kafka server. • Topic In Kafka, topic is a logic concept and is used to categories messages. The notion of topic is similar to tables in a database or directories in a file system. • Partition A partition is the elementary storage unit that stores messages of topics. A partition is a log file where records are written in an append-only fashion.

An Efficient Disaster Recovery Mechanism

299

• Offset Each record in a partition is assigned with a unique id called offset. When a record is produced and appended to a partition, Kafka increments the offset counter by one and then assigns it to this new record. To the consumer side, Kafka remembers the reading offsets which is the progress of message reading in a partition by keeping track the offset of the last message consumed. One partition can be consumed by multiple consumers, each with different reading offsets. • Consumer group A consumer group is a set of consumer processes that uses one reading offset for a partition. That is, each partition is processed by exactly one consumer in the group. With this mechanism the Kafka application can split workload to multiple consumers. Also, the messages can process in parallelize at high throughputs. A Kafka cluster consists of several brokers. The partitions and their replicas are distributed on brokers equally. 2.2 Multiple Kafka Clusters We can deploy multiple Kafka clusters to different geographical regions for serving data with locality. The messages produced from a region can be stored to the local cluster to save communication cost. The approach using multiple clusters enables cross-region backup which can be used to perform disaster recovery when a cluster crashes. When using cross-region backup between two clusters, the amount of messages are doubling, one for the original and the other for the copies. There are two modes for cross-region backup. The first is the active-active mode. Both of clusters are equal and two-way mirroring is used between the two clusters. A consumer reads data from both clusters at the same time, acting like a cross-cluster consumer group. In the active-active mode, in the event of a cluster failure, since the Kafka stream is available in both regions and contains the same data, the consumer can switch to the other region. This mode reduces the downtime to near zero and enables automatic failover when service fails [2, 3]. Another is the active-passive mode which is a one-way mirroring between two clusters. Data is mirrored asynchronously from the active cluster to the passive cluster. This mode tracks the consumption progress, i.e., offset, of the main region (active side) and makes copies to the passive side. If the main region fails, the consumer will switch to the passive region and resume the consumption progress according to the offset [1].

3 System Architecture Design In this section, we focus on designing a bidirectional backup mechanism between two regions. We discuss how to resolve and recover from cluster crashes and use data redundancy to support regional failover.

300

L.-P. Chen et al.

Fig. 2. The diagram of system architecture.

3.1 System Architecture In this section, we will introduce the disaster recovery model of this paper. Figure 2 shows the system architecture. Our method is to perform bidirectional backup in two Kafka clusters. For simplicity, Fig. 2 only shows clusters of two regions called Region A and Region B. In our model, partition is the basic unit of backup and recovery. We maintain a configuration file to store the ids of the partitions that need to be backed up and the service endpoints of Kafka clusters in region A and B. In this model, each region involves two additional servers called producer gateway and consumer gateway to perform the backup and failover, described in the following subsections. 3.1.1 Producer Gateway A producer does not connect to Kafka server directly. Instead, it connects to the producer gateway. The gateway provides communication transparency by hiding backup steps from the producers. A producer gateway performs the following functions: • Original message publishing: It simply forwards the original message to the destination. If the destination Kafka server fails, it blocks this action until the Kafka server is recovered. • Backup message publishing: The gateway will make a copy of the message and publish to the mirror Kafka servers in other regions. If a mirror server fails, it blocks this action until the mirror server is recovered. • Respond to the producer: If all of original and backup messages are failed, the gateway responds to the producer with a failed code, otherwise, a success code.

An Efficient Disaster Recovery Mechanism

301

3.1.2 Consumer Gateway Similar to the side of message production, a consumer does not directly connect to Kafka broker to read messages. It connects to the consumer gateway which serves as an intermediate node that supports backup and failover functionalities by hiding details steps to the normal consumers. We implement the consumer gateways by using the active-active mode as mentioned above. Each cluster has a consumer gateway that connects to other Kafka clusters. Thus, for n cluster, there will be n gateways, each with n-1 connections. Each gateway runs two threads, one for monitoring the status of the system, and another for maintaining offsets, as described as follows. The action of a consumer gateway is much more complex than that of producer gateways. The processes of consumer gateway are described as follows. a) Monitoring Status of Clusters First, consumer gateway has to trace whether a Kafka cluster work normally or is closed. Additionally, the consumer gateway will periodically publish and reads dummy test messages to different regions to detect their congestion status and transmission delay. There are different timings for submitting test messages. When there are normal messages sent between two clusters during a predetermined time period, the system can analysis the transmission delay via normal messages. It is unnecessary to send test messages. On the contrary, if there was no communication between the two for a long time, the consumer gateway will send the test messages periodically to maintain awareness of system status. b) Maintaining Offsets Kafka maintains an offset for each pair of consumers (or consumer group) and partition. The offset represents the last position in message queue of the partition that consumer reads. When the consumer reads a message, the offset is then moved to the next position. In our system, there are three components that will keep track of offset related values: • Consumer application, • Consumer gateway, and • Kafka cluster The offsets stored in a consumer gateway are usually equal to that in Kafka cluster. When the Kafka cluster fails, the records of offsets remain keeping in the consumer gateway which can be used to resume the progress of the consumer application in another cluster. As shown in Fig. 3, producer 1 (or producer 2) produces messages Msg A (or Msg B) in Region A (or Region B). Assuming that the current progress of the consumer A is at the offset 3 (red) i.e., message A4 in Region A. Assume that in this time Region A crashes, after failover from Region A to Region B, the offset record of the consumer

302

L.-P. Chen et al.

will directly correspond to the offset of message A4 in Region B Quantity 3 (blue), and then compare it with the last stroke of the consumer’s consumption message. If they are the same, move the offset back one stroke, that is, offset 4 (red); otherwise, move the offset back one stroke until the comparison is correct. c) Remove Obsolete Messages This process periodically removes the obsolete replicated data stored in the replicate clusters. Since a cluster fails in low probability usually, if there is no failure for a long time, removing obsolete data in the replicated servers can save a lot of memory and disk storage space. d) Consumer Gateway Algorithm We implement the consumer gateways by using the active-active mode as mentioned above. In this mode, each cluster has a consumer gateway that connects to other Kafka clusters. The consumer gateway is described in Consumer_Gateway_Algorithm below. Thus, for n cluster, there will be n gateways, each with n-1 connections. Each gateway runs two threads, one for monitoring the status of the system, and another for maintaining offsets, as described as follows. The algorithm consists of four components that run on the consumer gateway server in a cluster, say A. The steps of four components are described in this algorithm and are explained in detail as above. The first component is a monitoring process that monitors the status of all other clusters. It keeps track of next candidate cluster to be connected when the current cluster fails. The second component is a process recording the last offset the consumer reading the message queue in the data partition. If a failover occurs, the consumer gateway automatically resumes the reading progress of the message queue in the new host cluster. The third process periodically removes the obsolete replicated data stored in other clusters. In the case of no failure for a long time, this mechanism will save a lot of memory and disk storage space. The last process performs the failover when the host cluster fails. In this process, the gateway server automatically switches to connect to another cluster and fix the offset to reflect the most recent progress. The Fig. 3 illustrates the mapping of offsets at failover from Region A to Region B.

An Efficient Disaster Recovery Mechanism

303

Algorithm Consumer_Gateway_Algorithm() 1. Monitor the status of clusters (1) Periodically publish TEST_MSG and read them back from the other Kafla clusters (2) Analysis the transmision delay between cluster A and other clusters (3) Find the cluster most approximated to cluster A. 2. Maintain offsets (1) Upon each message sent to this gateway, do: Record LAST_OFFSET for the pair of (user,partition) 3. Remove obsolete data: (1) Periodically send REM_OBS_MSG to other Kafla clusters Attach LAST_OFFSET in REM_OBS_MSG Notify the other cluster to remove the messages with offset older than LAST_OFFSET 4. Failover: (1) When the gateway detects the crash of its hosted cluster, do Re-connects to most approximated cluster, say B. Move the offset of the replicated partition in cluster B to LAST_OFFSET

3.1.3 Recovery of a Kafka Cluster After monitoring the resurrection of the local broker, confirm the remaining surviving topics and their offsets in the broker and compare the data with the brokers in other areas, and then use the consumer to consume the lost information to any other area, and the producer will produce the lost information back. The original broker. In short, grab the lost information from other areas and throw it back into the broker to achieve the effect of data copy. 3.1.4 Recovery of Kafka Broker A Kafka cluster consists of several Kafka brokers. In the cluster, a partition P and its replicas are distributed on several brokers to achieve fault tolerance. Among these brokers, one is elected as the leader broker in terms of P. Those non-leaders are termed followers. The leaders and followers work in a master-slave manner. Data writing and consumption are only performed by the leader broker. That is, when a producer sends messages to any follower broker that stores partition P, all the messages will be forwarded to the leader. Also, all the consume requests will be forwarded to the leader broker. Based on the replication configuration set by the producer e.g., replication time and the number of replicas, the leader periodically sends the copies of data to other followers using the backup protocol.

304

L.-P. Chen et al.

Fig. 3. The offset mapping from Region A to Region B.

In a Kafka cluster, if a non-leader broker fails, the destruction of this broker will not cause any message loss or consumption failure, since there are copies stored in the leader as well as other followers. On the contrary, when a leader broker fails, the FLE (i.e. Fast Leader Election) mechanism of zookeeper will be triggered to rapidly elect a new leader. Nodes are in the election stage at the beginning. As long as a node gets more than half of the votes of the nodes, it can be elected as the leader.

4 Experiments We build a disaster recovery mechanism for Kafka clusters. First, we install Docker in an Ubuntu environment and use Docker Swarm to connect clusters in different regions to each other. Then we build Docker Compose for container management, and then deploy Kafka in Docker Swarm. As illustrated in Fig. 4, we construct an environment which contains two clusters, Region A and Region B. Initially, at time 0, the consumer consumes messages from the broker in Region A at outbound rate around 110. The outbound rate represents BytesOutPerSec i.e., the rate of reading the number of messages per second.

An Efficient Disaster Recovery Mechanism

305

Fig. 4. The message outbound rate, that is, the Fig. 5. The message outbound rate at four rate at which the consumer consumes represents regions. The red, blue, pink, yellow from the the outbound rate of region A (or region B). broker. The red (or blue) line. colored lines represent the message outbound rate at Region A, B, C, D, respectively.

In the experiment shown in Fig. 4, at time 80 s, the cluster in Region A fails and the outbound rate goes down to 0 almost immediately. It stops at offset 74 at this time. In the meantime, the cluster in Region B takes over the service seamlessly. It continuous reads the data with offset 75 and the outbound rate for Region B goes up to 110 which almost the same as that of Region A. Figure 5 demonstrates another environment which contains four clusters, Region A, B, C and D. Initially, at time 0, the consumer consumes messages from the broker in Region A at outbound rate around 110. At time 80 s, the cluster in Region A fails and the outbound rate goes down to 0. In the meantime, the cluster in Region B takes over the service seamlessly. Also, the clusters in Region C and D are not affected by the failover with the stable outbound rate.

5 Conclusion This paper designs and develops the producer gateway and consumer gateway that provides a simple way for achieving disaster recovery and maintains progress for the client applications during failover. These gateway servers hide the intricate details behind and thereby achieve location transparency and fault tolerance for Kafka clusters. The experimental work demonstrates the successful results of our works. Acknowledgments. This study is financial support in part by Ministry of Science and Technology, Taiwan under the grants MOST 108-2221-E-029-009 and MOST 109-2221-E-029-017-MY2.

306

L.-P. Chen et al.

References 1. Apache Kafka. https://kafka.apache.org/documentation/ 2. Active-active and active-passive failover. https://docs.aws.amazon.com/Route53/latest/Develo perGuide/dns-failover-types.html 3. Active/Active Architecture. https://docs.cloudera.com/csp/2.0.1/srm-overview/topics/srm-act ive-active-arch.html 4. Soman, C.: uReplicator: Uber Engineering’s Robust Apache Kafka Replicator (2022). https:// eng.uber.com/ureplicator-apache-kafka-replicator/ 5. Apache Kafka’s MirrorMaker. https://docs.confluent.io/4.0.0/multi-dc/mirrormaker.html 6. uReplicater. https://www.163.com/dy/article/G1E7A18A0511D3QS.html

Detection and Defense of DDoS Attack and Flash Events by Using Shannon Entropy Shih-Ting Chiu1 , Heru Susanto2,3 , and Fang-Yie Leu1,4(B) 1 Computer Science Department, Tung-Hai University, Taichung City, Taiwan

{G08350023,leufy}@thu.edu.tw

2 National Research and Innovation Agencies, Jakarta, Indonesia 3 University of Technology Brunei, Darussalam, Brunei

[email protected] 4 Emergency Resource Management Center, Ming-Chun University, Taoyuan City, Taiwan

Abstract. Nowadays, Hackers are everywhere. Current information systems often suffer from attacks anytime. Also, 5G network have gradually entered human lives to satisfy our network-service needs. At present, without human intervention, 5G networks still lack tools to automatically detect attacks and identify flash events. Many scholars and experts have proposed mechanisms to defend DDoS attacks, including detecting attacks and mitigating the damage caused by attacks. However, there is no security mechanism to defend DDoS attacks in the 5G networks. In other words, the existing defense systems cannot effectively protect 5G users and their infrastructures. Therefore, in this research, we present a system called Detection and Defense of DDoS on 5G (DDD5G for short), which uses the average entropy generated at different time intervals as the threshold, and then compares it with the entropy of current traffic. After simulating on MiniNet, we confirm that DDD5G detect simultaneous attacks, i.e., it can effectively self-detect and defend DDoS attacks without human intervention.

1 Introduction In recent years, the frequency that hackers attack information systems has been increasing. The European Telecommunications Standards Institute (ETSI) has developed 5G specifications. Although its system architecture has taken into account the system level security, e.g., 5G SA (Stand-alone) and 3GPP R15 define stronger security capabilities, including adding service domain security, adopting the security mechanisms and security protocols of registration, function discovery, user authorization, etc., to guarantee the security of the 5G Service-Based Architecture (SBA), etc. But, security of 5G network still needs to be enhanced. It also utilizes a unified authentication framework to integrate different forms of multiple access authentication methods. i.e., 5GAKA, EAPAKA and EAP-TLS, which are applied to transmitted data to enhance data privacy e.g., SUCI (Subscription concealed Identifier) is sent to UDM (Unified Data Management) during authentication as supporting user plane data integrity protection. Further, hackers may illegally track the user’s location and information. R16 and R17 go a step further by providing more comprehensive security capabilities, e.g., the SBA architecture, to © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 307–314, 2022. https://doi.org/10.1007/978-3-031-08819-3_32

308

S.-T. Chiu et al.

guarantee the security of data transmission in the control plane and user plane in a 5G core network(5GC) [1]. On the other hand, 5G network traffic delivery consists of various protocols and connections, resulting in broader and more serious security threats. In order to make up for the shortcomings of the current 5G security protection and prevent 5G networks from DDoS attacks, in this research, we propose a network autonomous security system called Detection and Defense of DDoS on 5G (DDD5G for short), which adopts Shannon entropy to detect whether a 5G network is under DDoS attacks or not. When a DDoS attack occurs, it provides self-determined methods to disconnect the connections so as to mitigate the impact of the attack on users. The DDD5G can also distinguish flash events that are partially similar to DDoS so that users who generate flash events can continue to receive normal services from the system. As a results of our experiment, the DDD5G functions are feasible, and it can indeed effectively defend against DDoS attacks without human intervention. This paper is organized as follows. Section 2 reviews related literature of this study. Section 3 introduces the function of each component of this system. Functional simulation and verification are presented and discussed in Sect. 4. Section 5 concludes this study and addresses our future work.

2 Related Work 2.1 Related Studies Currently, most network security systems have tried to improve the rules of their firewalls, or to strengthen the performance of their equipment. However, they are not self-managed management systems which mitigate the damage caused by DDoS attacks. Virmani et al. [2] proposed an entropy deviation method for analyzing network intrusion, and detecting malicious behaviors issued by hackers. This method analyzes the level of entropy deviation, with which to determine whether a user’s network behavior is abnormal; Sahoo et al. in [3] presented a 5G network, in which SDN controller is used to count traffic, and General Entropy is employed to detect DDoS attacks at its control plane. Daneshgadeh et al. [4] mixed Shannon Entropy and KOAD & Mahalanobis Distance to identify whether the protected system is under DDoS attacks. This system can more accurately distinguish DDoS and flash events. Mamolar et al., [5] raised an autonomous security system, which is the management network set up by the authors, to decide whether a system is under DDoS attack. If yes, the system will initiate countermeasures without user intervention. It effectively defends against DDoS attacks. The main network component responsible for detecting DDoS attacks is edge computers. 2.2 Intrusion Detection System (IDS) Figure 1 shows a reference example of NIDS deployment. NIDS can be installed on the server side of the firewall or on the Internet side. The benefit of the latter is that the location is exactly the throat point of the network. It is able to see all the various attack methods that attackers from the Internet will apply to the system. The advantage of the

Detection and Defense of DDoS Attack and Flash Events

309

former is that packets are first checked by the firewall, and the packets that violate the firewall policies are discarded, thus reducing the burden of NIDS in detecting packets, and focusing on the detection and analysis of attacks. NIDS can also be protected by the firewalls. This will reduce the chance of being attacked.

Fig. 1. Example locations of NIDS deployment

2.3 5G System Architecture Mobile Edge Computers are usually placed near their users to serve the needs of nearby users, and to reduce the long transmission delay caused by the long-distance transmission, which prolongs the system response time. 5G Core Network (5GC) usually installed in the computer room. The main function is user authentication and operation control. It also allows users to access other networks through 5GC settings. 2.4 Shannon’s Entropy and Network Defense In information theory, network entropy is used to reflect the probability of different degrees of uncertainty in a system. In fact, the increase or decrease in the amount of information is uncertain to the user. Shannon [6] expressed the degree of information uncertainty with the concept of thermodynamic entropy, which is called information entropy. Suppose there is a discrete random variablex = {x1 , x2 , x3 , . . . , xn }, which has n distinct values. The definition of p(xi ) is the probability that the random variable’s value isxi . The entropy H (x) of the random variable x is defined as H (x) = −

n  i=1

p(xi )log2 p(xi )

(1)

310

S.-T. Chiu et al.

3 The DDD5G 3.1 Network Defense Mechanism Figure 2 shows the schematic diagram of the DDD5G. Packets, denoted by P, are sent from mobile phones to the base station. Before transferring P to switches, the DDD5G duplicates P, denoted by P and sends P’ to EC for intrusion detection. If the detection is abnormal, it is further determined whether it is an attack. If yes, the DDD5G stores the packet information in the alarm Index and notifies the SDN controller to update the flow entry or entries of the flow table in this switch. This switch will drop all packets sent by the resource IP of P to prevent it from continuing harming the protected system. If it is determined that the transmission is a flash event, the flow entry or entries for routing packets sent by this IP address will not be changed. So the switch continues to transmit the packets sent by the source IP to their destinations.

Fig. 2. Architecture of DDD5G network defense mechanism

3.2 Our Detection and Defense Algorithm Our detection and defense algorithm as shown in Fig. 3 collects the flow information of incoming packets in time units of t1 seconds, calculates their entropy as the threshold, denoted by TH, and then stores the information in the Flow index created in the DDD5G. Here, t1 = 4 s. Let SE be the entropy calculated on the packets collected during t2 . If TH ≤ SE, it will be considered as normal traffic. Otherwise, SE will be judged as abnormal, and next judge will be conducted by continuing collecting following traffic for t3 sec and calculating the traffic’s entropy, e.g., denoted by RD. If TH ≤ RD, it will

Detection and Defense of DDoS Attack and Flash Events

311

be determined to be flash event. Otherwise, the corresponding flow entries of underlying flow table will be changed, to drop all packets sent by this source IP. The source IP will be remarked as a hacker and stored in the abnormal index.

Fig. 3. The DDD5G algorithm

4 Experiment and Validation The experiment is performed in the following to verify the autonomous security of the DDD5G. First, the server S is attacked with a large flow issued by single user; second, the server S is attacked by issuing multiple small flows. 4.1 Attack Description We conduct DDoS attack on Mininet [7] using SDN Controller RYU to control SDN switches. UEs send flooding packets to attack Victims.

312

S.-T. Chiu et al.

4.2 Experiment Platform The topology as shown in Fig. 4 consists of 8 hosts as traffic senders. h1, h2, h3 and h4 transmit packets to their destinations through S1 switch. h5, h6, h7 and h8 send packets to their targets through S2 switch. All packets are duplicated to the C0 controller to capture packet characteristics. In C0 controller, the DDD5G algorithm classifies packets and detects abnormal traffic. When there is an abnormal, packets sent by the identified hacker will be dropped by C0. This hacker’s IP address will be stored in the abnormal index for IP comparison when C0 is trying to identify malicious behaviors of the following incoming traffic.

Fig. 4. Experimental topology

4.3 Functional Verification Results At the beginning of this experiment, we establish the experimental topology and compute TH value. The experimental result is shown in Fig. 5. When normal packets are sent, TH < SE. The normal information is displayed on the terminal. When the DDoS attack is issued, TH ≥ SE. The DDD5G determines that the protected system is under attack.

Detection and Defense of DDoS Attack and Flash Events

313

Fig. 5. Experimental result

5 Conclusion and Future Studies This research proposes an architecture that can autonomously protect 5G networks from DDoS attacks. Once attacked by DDoS, it can mitigate the damage suffered by the users. The packets sent by UE are duplicated by using mirror port of a switch to the edge computers to extract packet characteristics. Shannon Entropy is used to detect DDoS attacks and identify flash events so as to allow normal users to continue receiving network services. The DDD5G is placed in a logical position, e.g., close to the switch or attack source (e.g., UE) to block attack packets much earlier, thereby reducing the damage that the victim suffers. The experiments show that once the protected system is attacked by Bot-net, the self-managing loops will trigger the SDN controller to immediately reduce attack traffic sent to the victim in about one second. That means, the DDD5G effectively reduces the bandwidth occupied by attack packets. However, the extensibility of the DDD5G topology is slightly insufficient. There is a need to collect normal traffic in advance to calculate the threshold. It will bring forth a higher detection delay to the system. In the future, we will improve it by means of machine learning. We will also derive the reliability and behavior patterns of the DDD5G so that users can realize the system’s reliability and behavior before using it. These constitute our future research directions. Acknowledgments. This study is financial support in part by Ministry of Science and Technology, Taiwan under the grants MOST 108-2221-E-029-009 and MOST 109-2221-E-029-017-MY2.

314

S.-T. Chiu et al.

References 1. 5G security: IMT-2020(5G) promotion, China Academy of Information and Communications Technology. https://pdf.dfcfw.com/pdf/H3_AP202112141534538455_1.pdf?163950140 1000.pdf 2. Virmani, D., Taneja, S., Chawla, T., Sharma, R., Kumar, R.: Entropy deviation method for analyzing network intrusion. In: International Conference on Computing, Communication and Automation (ICCCA), April 2016, pp. 515–519 (2016) 3. Sahoo, K.S., Sahoo, B., Vankayala, M., Dash, R.: Detection of control layer DDoS attack using entropy metrics in SDN: an empirical investigation. In: The Ninth International Conference on Advanced Computing (ICoAC), December 2017, pp. 281–286 (2017) 4. Daneshgadeh, S., Ahmed, T., Kemmerich, T., Baykal, N.: Detection of DDoS attacks and flash events using shannon entropy, KOAD and Mahalanobis distance. In: 22nd Conference on Innovation in Clouds, Internet and Networks and Workshops (ICIN), February 2019, pp. 222– 229 (2019) 5. Mamolar, A.S., Pervez, Z., Calero, J.M.A., Khattak, A.M.: Towards the transversal detection of DDoS network attacks in 5G multi-tenant overlay networks. In: Computers and Security, November 2018, vol.79, pp. 132–147 (2018) 6. Information Entropy, Wikipedia. https://en.wikipedia.org/wiki/Entropy_(information_theory) 7. Mininet. http://mininet.org/

5G Base Station Scheduling Yi-Cheng Jian1 , Meng-Shao Chung1 , Heru Susanto2,3 , and Fang- Yie Leu1,4(B) 1 Computer Science Department, Tunghai University, Taichung City, Taiwan

{S07351059,S07351024,leufy}@thu.edu.tw

2 National Research and Innovation Agencies, Jakarta, Indonesia

[email protected]

3 University of Technology Brunei, Darussalam, Brunei 4 Emergency Response Management Center, Ming-Chun University, Taoyuan City, Taiwan

Abstract. 5G base stations (BS) distribute resources to User Equipments (UEs) by dividing the BS’s spectrum into sub-channels of different sizes, and then allocate them to UE’s flows for uploading or downloading data based on time length, which may be a long or short duration. In a 4G-network BS, the spectrum of a resource block (RB) is 180 kHz, while in 5G, a BS spectrum is divided into numerologies, the size of which may be: 15, 30, 60, 120, and 240 kHz. The BS algorithm Scheduling and Resource Allocation (SRA) algorithms implemented in this study include Proportional Fairness (PF), Maximum-Largest Weighted Delay First (M-LWDF) and Exponential/Proportional Fairness (EXP/PF). We evaluate the performance of these algorithms in handling different types of packets, e.g., Real-Time (RT) flow, Non-Real-Time (NRT) flow, etc. The SRA algorithms are developed in the 5G-air-simulator, which is an open source C++-based simulator, to deliver eMBB, uRLLC and mMTC packets for analyzing performance of different applications. The parameters used include BS transmission speed, flow size, etc. The metrics parameters include packet delays and transmission fairness, etc. The goal is to use the most suitable SRA to transmit different types of packets for building independent network slices with different service characteristics. Keywords: 5G new radio (NR) · Downlink scheduling algorithm · 5G-air-simulator · QoS · Scheduling and Resource Allocation (SRA)

1 Introduction The International Telecommunication Union (ITU) divides mobile network applications into three types, including (1) Enhanced Mobile Broadband (eMBB), e.g., carring AR/VR flows; (2) Ultra-reliable and Low Latency Communications (uRLLC), e.g., supporting industrial automation; (3) Massive Machine Type Communications (mMTC), e.g., delivering IoT traffic. In 5G networks, because the bandwidth that a Base Station (BS) can provide is limited, the data transfer between BS and User Equipment (UE) is one of the biggest challenges for high quality data flow. Even though carry aggregation can increase the BS service bandwidth, there are still many technical bottlenecks. On the one hand, many technologies need to be conquered before resources allocation of © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 315–324, 2022. https://doi.org/10.1007/978-3-031-08819-3_33

316

Y.-C. Jian et al.

5G BS can be optimized. For example, the 5G BS spectrum is currently divided into numerologies which may be 15, 30, 60, 120 and 240 kHz. But there are still technical difficulties to support the upcoming network slicing after 5G is launched. This study modifies the packet scheduling algorithm, i.e., the Scheduling and Resource Allocation (SRA), used in 4G LTE so that it can be applied in a 5G environment. We also add packet delivery priority which may be Real-Time (RT) flow or Non-Real-Time (NRT) flow. In principle, the former is given a higher priority. In order to observe the spectrum efficiency of BS using different packet scheduling algorithms, we adopt three downlink scheduling mechanisms, which are integrated with network slicing for resource allocation, and dynamically adjust the wireless resource allocation policies by referencing to the QoS parameters of UE, and the number of available RBs before providing the corresponding services to cellular phones. Finally, we experimentally analyze the mentioned SRAs given different parameters and their values. This paper is organized as follows. Section 2 reviews relevant literature of this study. Section 3 introduces the designed system of this study. Section 4 presents the simulation results and discussion. Section 5 concludes this study and addresses our future research.

2 Literature Review 2.1 Introduction to Algorithms The 5G SRA aims to maximize system performance so as to meet the service needs of all users, while taking into account spectrum utilization and performance fairness. 5G systems, like 4G networks, also introduce the concept of channel-sensitive scheduling. Channel quality is measured first and then the modulation and demodulation methods, such as QPSK or 512-QAM, are selected based on the channel quality. As shown in Fig. 1, SRA algorithms of a BS can be divided into QoS unAware (e.g., Proportional Fairness (PF)) and QoS Aware (e.g., Delay aware with Maximum-Largest Weighted Delay First (M-LWDF) and Exponential/Proportional Fairness (EXP/PF), etc.).

Fig. 1. Classification of packet scheduling algorithms [1].

5G Base Station Scheduling

317

2.1.1 QoS-unAware PF The typical QoS-unAware algorithm is the proportional fairness algorithm [2] suitable for the transmission of NRT flow. It considers channel quality and throughputs previously experienced by the user to balance system throughputs and effectiveness of resource allocation for achieving performance fairness. It does not guarantee any quality of service. In Eq. 1,wi,j (t) is the weight of allocating resource block (RB) RBj to flow i of UE. wi,j (t) =

ri,j (t) Ri (t)

(1)

where ri,j (t) is defined as the instantaneous rate transmitted by flow i at time t through RBj [3], and Ri (t) is defined as the average transmission rate of flow i up to time t (see Eq. 2): Ri (t) = [(1 − β) ∗ Ri (t − 1)] + [β ∗ ri (t − 1)]

(2)

where, β is a parameter that smooths system transmission rate and controls system fairness; ri (t) is the instantaneous transmission rate of flow i at time t. . 2.1.2 QoS-Aware There are many existing QoS algorithms. This study only addresses M-LWDF and EXP/PF. A. M-LWDF The M-LWDF algorithm [4] adopts the PF algorithm to deliver NRT flows. When allocating resources to RT flow, the packet delay and the delay time of the head-of-line packet in data queue are important parameters. However, the packet delay is more serious in a non-continuous reception mode. Thus, the best quality of service cannot be guaranteed if only the head-of-line packet delay is considered. Let wi,j (t) be the weight of allocating RBj to flow i at time t; Let Ri (t) be the average transmission rate of flow i at time t; let ri,j(t) be the instantaneous transmission rate of flow i at RBj at time t. wi,j (t) is defined as ⎧   ri,j(t) ⎨α D i HoL,i (t) R (t) , if i ∈ RT i wi,j (t) = (3) ri,j(t) ⎩ , if i ∈ NRT Ri (t)

ai = −

log δi τi

(4)

where τi is defined as the delay threshold of flow i; δi denotes the maximum probability that the head-of-line packet delay of flow i exceeds τi in the queue. B. EXP/PF The EXP/PF algorithm [5] can be used to support multimedia delivery for Adaptive

318

Y.-C. Jian et al.

Modulation and Coding (AMD) and Time Division Multiplexing (TDM) systems, either for RT or NRT services for a single user. For each user’s service type, the EXP/PF weight wi,j (t) is defined in Eq. 5. ⎧   ⎨ exp αi DHoL,i√(t)−h(t) ri,j (t) , if i ∈ RT 1+ h(t) Ri (t) wi,j (t) = (5) ri,j(t) ⎩ , if i ∈ NRT Ri (t)

where h(t) is defined in Eq. 6, which indicates the delay of the overall RT flow of the system, and Ni is the number of RT flows currently in the system. This algorithm aims to reduce the delay of all RT flows. h(t) =

1 Ni Σ αi DHoL,i (t), for i ∈ RT Ni i=1

(6)

3 System Architecture In this research, the M-LWDF algorithm is able to transmit RT flows. It delivers NRT packets with PF algorithm. CBR (mMTC) packets are set as lower priority NRT flow, and mMTC is usually used to transmit IoT packets. The PF algorithm estimates the average transmission rate of UE before assigning RB to UE. Video (eMBB) and VOIP (uRLLC) packets are delivered via RT flows calculated by using Eqs. 3 and 4. If the delay time of the first packet in the queue exceeds the delay threshold, it is discarded to avoid wasting bandwidth resources, i.e., to avoid the occurrence of the head-of-line blocking problem. The EXP/PF algorithm improves and inherits the advantages of the PF and M-LWDF algorithms. The PF algorithm is employed when processing NRT-flow packets. Due to inheriting the M-LWDF algorithm, it also determines whether the delay time of the first packet in the queue exceeds the delay threshold or not when processing RT flows. However, the M-LWDF may ignore the expiring RT flow packets, e.g., P, inside the queue. The EXP/PF algorithm, referring to Eqs. 5 and 6, removes P in the queue. When using 5G-air-simulator as our simulation tool, after selecting a 5G environment, packets are generated and transmitted from BS to UE. Also, it confirms packet type, i.e., eMBB, mMTC or uRLLC, of a receiving packet and delivers it via PF, M-LWDF or EXP/PF. We then observe there performance. Figure 2 lists our flowchart.

5G Base Station Scheduling

319

Fig. 2. Research flowchart

4 Simulation Results and Discussion 4.1 Simulation Scenario Table 1. Simulation parameters Parameters

Values

Carrier frequency

2.1 GHz

System bandwidth

20 MHz

Number of cells

7

Number of serve BSs

1

Number of RBs

100

Number of UEs

10–60

UE speed

3, 30, 120 km/h

UE distribution

Random

Max delay

100 ms

Frame structure

FDD

Cell radius

1 km

Traffic model

VoIP, Video, CBR

Video bitrate

1000, 7000, 17000 kbps

Flow duration

25 s

Simulation time

30 s

Simulation repetitions per scheduler

10

SRA algorithm

PF, M-LWDF, EXP-PF

Table 1 lists the parameter settings of our simulations. In the Cellular system, seven cells are connected into a hexagonal shape and many UEs are connected to the main

320

Y.-C. Jian et al.

omnidirectional BS. The other BS do not provide any network services, only being used to generate small inter-cell interference. In the first experiment, each UE produces VoIP, Video and CBR Flows simultaneously. In the second experiment, the system performance of the UE is measured given different UE’s moving speeds. 4.2 QoS Evaluation In the first experiment, we evaluate Packet Loss Rates (PLRs), packet delays, throughputs and spectrum efficiencies of these algorithms on different types packets given different parameter values. Each experiment is repeated 10 times. VoIP, Video and CBR are individually transmitted with different numbers of UEs, ranging from 10 to 60. UEs’ moving speed is fixed 30 km/h. The PLRs of VoIP, Video and CBR are shown in Fig. 3. When the number of UEs increases, in general, the PLRs are higher. However, VoIP generates small traffic (see Fig. 3(a)) as a RT flow, and its PLRs are flat compared to Video (uRLLC) and CBR (mMTC), meaning that VoIP performance is not seriously affected by the number of UEs, or the transmission channel is not saturated. In addition, the PF also outperforms the other two schemes in terms of PLR due to the small flow size of CBR. When the sizes of Video Flows are larger, the two algorithms with QoS parameters outperform to the PF (see Fig. 3 (b)).

Fig. 3. Packet loss rates for VoIP, video and CBR flows.

As shown in Fig. 4, the PF algorithm does not have QoS and thus its PLRs are longer than those of the other two algorithms, no matter whether VoIP, Video or CBR is utilized. In other words, delay is one of the important parameters of QoS, aiming to reduce packet delivery latency and improve user experience.

Fig. 4. Delays for VoIP, video and CBR flows.

The throughputs of the three types of applications are shown in Fig. 5. As the number of users increases, the throughputs of all applications are all higher because more packets

5G Base Station Scheduling

321

are transmitted. The PF does not have QoS, its PLR is higher, about 90% when number of UE is higher than 20 (see Fig. 3(b)), thus resulting in lower throughputs compared with those of the other two algorithms.

Fig. 5. Throughputs for VoIP, video and CBR flows.

System spectrum efficiency is one of the performance indicators of network resource utilization. As shown in Fig. 6, the spectrum efficiencies of M-LWDF and EXP-PF increase to about 7 bits/Hz when the number of UEs is higher than 20. In this experiment, the flows of Video are the overall majority (see the y-axis dimension in Fig. 5(b)), so the curves illustrated in Fig. 6 are themselves similar to those of Video throughputs (see Fig. 5(b)).

Fig. 6. System spectral efficiencies.

Fig. 7. Packet loss rates for video flows on UE’s moving speeds, including 3, 30 and 120 km/h.

4.3 Different Moving Speeds of UEs In this experiment, when UEs move at different speeds, including 3, 30 and 120 km/h, and the simulated application is Video flows only. Each UE transmits Video flow packets at a fixed rate of 440 kbps.

322

Y.-C. Jian et al.

The PLRs plotted in Fig. 7 increase when UE moves and the number of UEs joining the BS is higher; the PLRs also increase when UE moves faster (compare the y-axis of the three figures illustrated Fig. 7 plots), e.g., when the UE’s moving speed accelerates from 3 km/h to 30 km/h, the PLRs of the PF are also higher because the PF only considers channel quality. The delays of video flow are shown in Fig. 8. The delays of the three algorithms are all less than 0.1 s when UE’s moving speeds are 3 km/h and 30 km/h and the numbers of UEs are 10 or 20. Figure 8 also indicates that the PF cannot control packet delays effectively.

Fig. 8. Delays for video flows on UE’s moving speeds, including 3, 30 and 120 km/h.

Fig. 9. Throughputs for video flows on UE’s moving speeds, including 3, 30 and 120 km/h.

Fig. 10. Spectral efficiencies for video flows on UE’s moving speeds, including 3, 30 and 120 km/h.

Figures 9 and 10 present the throughputs and spectral efficiencies respectively. Since only Video flows are employed, two arbitrary figures with the same moving speeds are similar, e.g., Figs. 9(i) and 10(i), i = a, b, or c. In addition, the throughputs and spectral efficiencies of the three algorithms decrease significantly when UE accelerates from 3 km/h (see Figs. 9(a) and 10(a)) to 120 km/h (see Figs. 9(c) and 10(c)) (please compare y-axis scales). Although, the models for simulating throughputs and spectrum efficiencies may not be 100% correct, the results

5G Base Station Scheduling

323

are still worth for reference. Based on the results of throughputs (spectral efficiencies), the three algorithms can barely solve the problem of high UE’s moving speed. Other methods such as predicting antenna [9], CSI prediction [8], etc. may be needed to solve this problem.

5 Conclusion and Future Prospects In this research, SRA algorithms with different QoS parameters, e.g., Delay unAware and Delay Aware, are explored to increase the difference between RT and NRT functions. The purpose is to improve the performance of different types of packets, and to use the corresponding SRA algorithms according to the customer’s needs, so as to increase the system capacity, and to create independent network slices with different service characteristics. One type of packet is transmitted via an independent slice. We also evaluate and analyze the performance of downlink scheduling algorithms, including packet loss rates, delays, throughputs and system spectrum efficiencies. The experimental results show that algorithms with Delay Aware, such as M-LWDF and EXP-PF, perform well in terms of throughputs and packet transmitting latencies. We will continue to investigate more SRA algorithms for 5G networks in the future, and analyze how to achieve the expected performance in different 5G environments to meet customer needs. We will also explore the challenges that a real BS may face so as to meet the needs of BS slicing. Acknowledgments. This study is financial support in part by Ministry of Science and Technology, Taiwan under the grants MOST 108-2221-E-029-009 and MOST 109-2221-E-029-017-MY2.

References 1. Vora, A., Kang, K.-D.: Effective 5G wireless downlink scheduling and resource allocation in cyber-physical systems. Technologies 6(4), 1–20 (2018) 2. Abdel-Hadi, A., Clancy, C.: A utility proportional fairness approach for resource allocation in 4G-LTE. In: International Conference on Computing, Networking and Communications (ICNC), Honolulu, HI, USA, pp. 1034–1040 (2014). https://doi.org/10.1109/ICCNC.2014. 6785480 3. Capozzi, F., Piro, G., Grieco, L.A., Boggia, G., Camarda, P.: Downlink packet scheduling in LTE cellular networks: key design issues and a survey. IEEE Commun. Surv. Tutor. 15, 678–700 (2012) 4. Xian, Y.J., Tian, F.C., Xu, C.B., Yang, Y.: Analysis of M-LWDF fairness and an enhanced M-LWDF packet scheduling mechanism. J. China Univ. Posts Telecommun. 18(4), 82–88 (2011) 5. Basukala, R., Ramli, H.A.M., Sandrasegaran, K.: Performance analysis of EXP/PF and MLWDF in downlink 3GPP LTE system. In: First Asian Himalayas International Conference on Internet, Kathmundu, Nepal, pp. 1–5 (2009) 6. Martiradonna, S., Grassi, A., Piro, G.: 5G-air-simulator: an open-source tool modeling the 5G air interface. Comput. Netw. 177, 107314 (2020)

324

Y.-C. Jian et al.

7. Kawser, M., Farid, H., Hasin, A., Sadik, A., Razu, I.: Performance comparison between round robin and proportional f air scheduling methods for LTE. Int. J. Inf. Electron. Eng. 2, 678–681 (2012). https://doi.org/10.7763/IJIEE.2012.V2.186 8. Hao, G.: Predictor antenna systems: exploiting channel state information for vehicle communications. Thesis of Chalmers University of Technology, Sweden, May 2020. arXiv: abs/2005.09758 9. Luo, C., Ji, J., Wang, Q., Chen, X., Li, P.: Channel state information prediction for 5G wireless communications: a deep learning approach. IEEE Trans. Netw. Sci. Eng. 7(1), 227–236 (2020). https://doi.org/10.1109/TNSE.2018.2848960

Asymmetric Cryptography Among Different 5G Core Networks Yu-Syuan Lu1 , Heru Susanto2,3 , and Fang-Yie Leu1,4(B) 1 Tunghai University, Taichung City, Taiwan

[email protected]

2 National Research and Innovation Agencies, Jakarta, Indonesia 3 University of Technology Brunei, Darussalam, Brunei

[email protected] 4 Emergency Response Management Center, Ming-Chun University, Taoyuan, Taiwan

Abstract. In 5G networks, packets transmitted from local UPF of a 5G network to another UPF in different destination 5G network is not encrypted. This conducts a risk of data leakage, particularly along the connection established between the two UPFs. To solve this problem, in this study, we propose an architecture that encrypts/decrypts packets with edge computers by using asymmetric cryptography for UEs.

1 Introduction In a 5G system (e.g., 5G-A), when UE (e.g., UEA ) wants to deliver data to another UE (e.g., UEB ) of another 5G system (e.g., 5G-B), UEA first informs its Access and Mobility Management Function (e.g., AMFA ) through RAN (e.g., RANA ), and then AMFA will inform Session Management Function (e.g., SMFA ) to establish a session UEA -RANA UPFA -Data Network (DN). The connection sequence built in 5G-B is opposite to that in 5G-A, i.e., DN-UPFB - RANB -UEB . Currently, data delivered between UPFA and UPFB is not encrypted, thus maybe maliciously accessed by hackers for, e.g., data tampering or eavesdropping. Therefore, this study, we would like to establish a secure channel to protect packets transmitted between UPFA and UPFB . Basically, for both systems, an edge computer (EC) is placed between UPFA and DN (between UPFB and DN) to encrypt data and decrypt data delivered between the two UPFs.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 325–333, 2022. https://doi.org/10.1007/978-3-031-08819-3_34

326

Y.-S. Lu et al.

2 Background and Related Work 2.1 The Security Mechanism Between UE-RAN-AMF

Fig. 1. 5G architecture.

The security mechanism between UE-RAN-AMF as shown in Fig. 1 includes data encryption and decryption, the connection and activities performed between UE and RAN (between UE and AMF) is called Access (non-Access) stratum, AS (NAS) stratum for short. When UE delivers a control message M to AMF via RAN, UE first encrypts M by using NAS key, producing M’, and then encrypts M’ to M” by using AS key. After receiving M” from UE, RAN decrypts M” to M’ by using AS key, and then delivers M’ to AMF. On receiving M’, AMF decrypts M’ to M by using NAS key. 2.2 5G-Authentication and Key Agreement (5G-AKA) The 5G-AKA [1] authentication process is as follows. 1. 2. 3.

4.

5.

6.

7.

UE delivers subscription concealed identifier (SUCI) or globally unique temporary UE identity (GUTI) as register request-1 (Reg Req-1) to RAN. As receiving the request, RAN adds the service-network name (SN-name) to the packet as Reg Req-2, and delivers it to AMF/Security Anchor Function (SEAF). SEAF makes sure whether this packet contains SUCI or GUTI. If it is SUCI, the packet will be passed to authentication server function (AUSF). If it is GUTI, SEAF will look for the matching subscription permanent identifier (SUPI) in SEAF. If yes, meaning that this UE has been authenticated, SEAF sends the packet containing SUPI and SN-name to AUSF. Otherwise, SEAF will request UE to resend a register request which includes SUCI, instead of SUPI. Of course, the process will go to Step (1). AUSF sends Reg Req-4 which includes (SUCI/SUPI, SN-name) to Unified Data Management (UDM)/Authentication credential Repository and Processing Function (ARPF). UDM/ARPF checks the SN-name contained in Reg Req-4 to make sure whether this base station actually belongs to present mobile network operators (MNO) or not. If not, the connection will be disconnected. UDM/ARPF further verifies whether Reg Req-4 contains SUCI or SUPI. If it is the former, UDM shall invoke Subscription Identifier De-concealing Function (SIDF) to de-conceal SUCI to SUPI. UDM/ARPF accesses the LTE Key from its database based on this SUPI.

Asymmetric Cryptography Among Different 5G Core Networks

8.

9.

10. 11.

12.

13. 14. 15. 16.

327

UDM/ARPF produces m sets of authentication vector (AV), m = 2 or 3, and executes the following procedures by invoking f1(LTE Key, (SQNRANDAMF)) to yield message authentication code (MAC), f2(LTE Key, RAND) to produce expected response (XRES), f3(LTE Key, RAND) to produce cipher key (CK), f4(LTE Key, RAND) to yield integrity key (IK), f5(LTE Key, RAND) to produce anonymity key (AK), UDM/ARPF also computes SQN ⊕ AK. UDM/ARPF invokes KDF(CKIK, SN-nameSQN ⊕ AK) to produce KAUSF . SQN ⊕ AKAMFMAC is named AUTN. RANDXRESKAUSFAUTN is called 5G Home Environment Authentication Vector (5G HE AV). The parameters above including RAND, AUTN, XRES and KAUSF are organized as an 5G HE AV. Because of generating m RANDs, m sets of AVs are consequently produced. UDM delivers Reg Res-4, which carries SUPI and m HE AVs derived from the m RANDs and corresponding parameters to AUSF. AUSF randomly chooses one of the HE AVs, and accesses this 5G HE AV’s RANDXRESKAUSFAUTN, keeps XRES and KAUSF at AUSF, hashes XRES into HXRES, produces KSEAF based on KDF (KAUSF , SN-name), and yields 5G Serving Environment Authentication Vector (5G SE AV) (includes RANDHXRESKSEAF AUTN). AUSF transports Reg Res-3 which carries 5G Serving Environment Authentication Vector (5G SE AV) (includes RANDHXRESKSEAF AUTN) to SEAF. SEAF accesses RANDHXRESKSEAF AUTN from this 5G SE AV, and keeps HXRES and KSEAF in its database. SEAF transmits NAS message Authentication-Request, which contains (RAND, AUTN, ngKSI, ABBA) to RAN. RAN delivers authentication request Reg Req-6 which conveys (RAND, AUTN, ngKSI) to UE. After acquiring this request, UE runs the following procedure with the LTE Key retrieved from its universal mobile telecommunications system subscriber identity module (USIM) and RAND sent by RAN, and invokes f1(LTE Key, (SQNRANDAMF)) to produce expected message authentication code (XMAC), f2(LTE Key, RAND) to produce response RES, f3(LTE Key, RAND) to produce CK, f4(LTE Key, RAND) to yield IK, f5(LTE Key, RAND) to generate AK, UE also calculates SQN⊕AK.

328

Y.-S. Lu et al.

17. UE retrieves MAC from AUTN, i.e., SQN ⊕ AKAMFMAC, and checks to see whether MAC and XMAC (generated by itself) are equal or not. If not, it. terminates this authentication process. Otherwise, 18. UE transports Reg Res-6 which carries RES produced by itself to RAN. 19. RAN sends Reg Res-5 which conveys RES to SEAF. 20. SEAF hashes RES received from RAN to generate HRES, and compares HRES and HXRES (see Step (13)). If they are not equal, it terminates the authentication. process. Otherwise 21. SEAF transmits Reg Res-7 to AUSF. Reg Res-7 includes RES and SUCI or SUPI. 22. AUSF checks the equality of the RES and the XRES (see Step (11)). If they are not equal, the authentication process will be terminated. Otherwise, 23. AUSF sends Reg Res-7 (which is produced for showing the success of delivery), SUPI and KSEAF to SEAF. 24. SEAF invokes KDF (KSEAF , (SUPIABBA)) to produce KAMF , and then sends KAMF to AMF, where SEAF and AMF coexist with each other; KDF (KAMF , NAS uplink count) to produce encryption/decryption keys, i.e., KgNB , where KAMF and KgNB are respectively the keys for encrypting/decrypting the subsequent messages delivered in AS stratum and NAS stratum. 2.3 Asymmetric Encryption and Decryption Assuming that KPub A (KPrv A ) is Alice’s public (private) key. Assuming that KPub B (KPrv B ) is Bob’s public (private) key. Alice and Bob individually publish their public keys [2]. When wishing to deliver message M to Bob, Alice encrypts M to KPub B (M), and delivers it to Bob. On receiving the KPub B (M) from Alice, Bob applies its own private key KPrv B to decrypt this encrypted message, i.e., M = KPrv B (KPub B (M)). When wanting to deliver message M’ to Alice, Bob uses Alice’s public key to encrypt M’ to KPub A (M’), and delivers it to Alice. Alice employs its own private key to decrypt the encrypted message to M’. 2.4 Message Authentication Code When Alice needs to deliver message M to Bob, Alice first calculates KINT (M) with a hash function, the key of which is KINT . Alice delivers MKINT (M) to Bob. When receiving this message, Bob retrieves M, performs hash function K’INT (M), and judges whether K’INT (M) = KINT (M)? If they are equal, meaning M has not been tampered by hackers during its transmission. Otherwise, it drops MKINT (M). 2.5 Timestamp in a Packet Before delivering message M to Bob, Alice retrieves current time from her system clock as timestamp T’ [3]. When M arrives at Bob, Bob checks to see whether T”- T’ ≤ T or not where T” is the time stamp when Bob receives M and T is the maximum tolerable time for M to travel from Alice to Bob. If T”- T’ > T, then Bob drops the packet since we consider it is a replay attack.

Asymmetric Cryptography Among Different 5G Core Networks

329

3 System Model In this study, asymmetric encryption and decryption is adopted, ECA (ECB ) needs to keep ECB ’s (ECA ’s) public key KPub B (KPub A ) announced by ECB (ECA ) beforehand. When UEA would like to send packet P to ECB , it delivers KPub B (P) to ECB . When receiving KPub B (P), ECB decrypts KPub B (P) by using its private key to decrypt KPub B (P) to P, i.e., KPrv B (KPub B (P)) = P. 3.1 Asymmetric Cryptography See Figs. 2a, 2b, 2c, 2d and 2e.

Fig. 2a. The process from UEA to ECA in Steps 1 to 5.

Fig. 2b. The process from ECA to ECB in Step 6.

1.

2. 3.

Before sending M to UEB , UEA delivers a control message C, which carries the IP address of UEB , to AMFA . But before sending C, UEA encrypts C to C’ with KAMFA , and encrypts C’ to C” with KgNBA where KgNBA is produced by using KDF (KAMFA , NAS Uplink Count) by AMFA . On receiving C”, RANA decrypts C” to C’ by employing KgNBA , and sends C’ to AMFA . AMFA decrypts C’ to C by using KAMFA .

330

Y.-S. Lu et al.

Fig. 2c. The process from UEA to ECA in Steps 7 to 9.

Fig. 2d. The process from ECA to UPFB in Steps 10 to 11.

Fig. 2e. The process from UPFB to UEB in Steps 12 to 16.

Asymmetric Cryptography Among Different 5G Core Networks

331

4.

From C, AMFA knows that UEA would like to deliver a packet to UEB . AMFA informs SMFA to establish a PDU session for FlowA from UEA to DN by requesting SDN controller to set flow entries in the flow tables of all UPFA s along the path. Assume that n UPFs are on the path, i.e., UPF(A) = {UPFA1 , UPFA2 , · · · , UPFAn } (usually it is the best path), UPFAn is the final UPF, and UPFA1 is first UPF. 5. After that, each UPF in UPF(A) is set a flow entry in its flow table to expand the routing path to ECA and ECA is connected to DN. In other words, when a packet arrives at UPFAn along the path of UPF(A), it will be sent to ECA through the expanded connection. 6. ECA transmits a control message D, which contains IPs of UEA , to ECB via DN. 7. UEA encrypts message M to M’ with AS key KgNBA , and transmits it to RANA . 8. After receiving M’, RANA decrypts M’ to M with KgNBA , and sends it to UPFA1 . 9. On receiving M, UPFA1 first matches the match fields (source IP, protocol TCP, port number…), and then passes M to UPFA2 . In the following UPFi , i = 2,3, …, n, does the same in sequence. At last, UPFAn delivers M to ECA . 10. After receiving M, ECA accesses its current system clock TA , and encrypts MTA to KPub B (MTA) by using ECB ’s public key KPub B . ECA generates MAC KINT (MTA) and transmits KPub B (MTA )KINT (MTA ) to ECB via DN. 11. From the source IP of M, ECB knows that M is sent by ECA in 5GC-A. Then it retrieves KPub B (MTA ), and decrypts it to MTA by using KPrv B of ECB . After that, ?

12.

13.

14.

15. 16.

it calculates KINT (MTA ), and verifies whether KINT (MTA )C = KINT (MTA )R , in which KINT (MTA )C is the result computed by ECB , and KINT (MTA )R is the message authentication code contained in the packet received. If they are not equal, ECB drops the packet. Otherwise, it retrieves TA from MTA , and checks to see whether TB -TA T or not, in which TB is the timestamp of 5GC-B when ECB receives this packet because of considering it as a replay attack. If not, ECB discards this packet. Otherwise, ECB transports M to UPFB , i.e., UPFB1 . After UPFB receives M, since UPFB does not realize how to proceed, it delivers a Packet-In message to SDN controller, requesting SDN controller telling it what to do next. SDN controller establishes a PDU session between UPFB and RANB . Assume that the path consists of m UPFs in sequence, i.e., UPF(B) = {UPFB1 , UPFB2 , · · · , UPFBm }, and is connected to RANB that is serving UEB . ECB sends M to UPFB1 . After UPFB1 receives M, it matches the match fields of its flow entries. Because flow tables in all UPFs in UPF(B) are set beforehand, M can be transmitted to RANB . RANB encrypts M to M” with KgNBB , and sends it to UEB . After UEB receives M”, it decrypts M” to M with KgNBB .

4 Security Analyses 4.1 Message Integrity Message integrity is to ensure security of transmitted data from being tampered. Our method is appending the K(M) to the data as MK(M) where K(M) represents that M

332

Y.-S. Lu et al.

is encrypted by using key K. If M is tampered to M’ on the way to its destination, the receiver R will encrypt M’ to K(M’), and then compare K(M’) and K(M) received. If they are not equal, its means M has been tampered. Both Steps 11 and 12 in Sect. 3.1 verify data integrity. That is when M is changed, R will drop this packet. 4.2 Replay Attack The replay attack represents that hacker H steals the message Q, Q = KPub S (MTS )KINT (MTS ), delivered from sender S to receiver R and impersonates S to send Q to R sometime later where TS is the time stamp when S sends Q to R and KPub S (MTS ’) is the integrity message. There are two situations. One is that H directly transmits Q to R without modifying Q. Due to multiple forwarding, TR - TS > T where TR is the time stamp when R receives KPub S (MTS )MTS , and T is the maximum time required by Q to travel from S to R. Therefore, it is considered to be a replay attack. Another one is that hackers modify TS to TS ’ so as to meet the expression TR - TS ’ ≤ T. But R will discard Q since KINT (MTS ’) = KINT (MTS ) where KINT (MTS ’) is the integrity message generated by R. It is considered as tampered data. 4.3 Confidentiality on Asymmetric Cryptography Users A and B individually generate their own private key KPrv , and announce their own public key KPub to the public. If hackers illegally acquire data without knowing KPrv , they are still unable to decrypt the data. Thus, the transmission security can be guaranteed. 4.4 Eavesdropping Attack Eavesdropping attacks are attacks in which hackers intercept data packets delivered in the concerned network. They then extract sensitive messages and their contents. After collecting some amounts of packets, they can analyze sensitive data from these packets, e.g., the user’s password of his/her credit card. In this study, we use asymmetric cryptography, to encrypt/decrypt messages. The key KPub A (KPub B ) is public, but the key KPrv A (KPrv B ) has not been revealed, so even if hackers know KPub A (KPub B ), they cannot use KPub A (KPub B ) to crack messages. The methods mentioned above can effectively prevent data from being stolen (eavesdropping attack). It is not easy for hackers to extract sensitive data from a message. In fact, it is hard for hackers to decrypt an encrypted message.

5 Conclusion and Future Studies Currently, messages M delivered to UPFB by UPFA via DN is not protected. M may be eavesdropped. Our approach provides asymmetric cryptography, to prevent M from being hacked. We append a message authentication code to M to avoid it from being tampered. A timestamp is also appended to M to reveal a replay attack. Further, EC is used not only to reduce the burden of a 5G system on data, encryption/decryption, and

Asymmetric Cryptography Among Different 5G Core Networks

333

authentication, but also to ensure the security of the system’s data transmission. In this study, asymmetric cryptography is employed. ECA (ECB ) uses asymmetric cryptography before M is sent to ECB (ECA ). In the future, we will also derive the behavior model and reliability model for the proposed system, so that users can know their behavior and reliability before using it. These constitute our future studies. Acknowledgments. This study is financial support in part by Ministry of Science and Technology, Taiwan under the grants MOST 108-2221-E-029-009 and MOST 109-2221-E-029-017-MY2.

References 1. ETSI: Security architecture and procedures for 5G System. document TS 33.501 V15.2.0, 3GPP, October 2018 2. Rivest, R., Shamir, A., Adleman, L.: A method for obtaining digital signatures and public-key cryptosystems. Commun. ACM 21(2), 120–126 (1978) 3. Li, G., Lai, C.: Platoon handover authentication in 5G-V2X: IEEE CNS 20 poster. In: IEEE Conference on Communications and Network Security (CNS), pp. 1–2 (2020). https://doi.org/ 10.1109/CNS48642.2020.9162271

The Impact of Integrating Board Games into Chinese Teaching in the Elementary School on Learning Efficiency -An Example of the Indigenous Fifth and Sixth Graders in a Remote Area of Nantou County Chiou-Shya Torng1 , Ho Hsiao-Yi2 , and Chai-Ju Lu3(B) 1 ASIA University, 500 Lioufeng Road, Wufeng, Taichung 41354, Taiwan

[email protected]

2 Nantou County Ren-Ai Elementary School, No. 5, Shannong Ln., Ren’ai Township, Nantou

County 546, Taiwan 3 ASIA University, 500 Lioufeng Road, Wufeng, Taichung 41354, Taiwan

[email protected]

Abstract. The main purpose of this research is to investigate the effects of applying board games on Chinese teaching in the Chinese class of the elementary school. One group pretest-posttest design of experimental research method was adopted to conduct this study, which involved fifteen students. The study of applying board games on Chinese teaching continued for ten weeks, one class a week. The research tools used in the study are Project for implementation of Remedial Instruction-technology-based testing, PRIORI-tbt and Applying Board Games in Chinese Learning Questionnaire. The nonparametric statistics were used by Wilcoxon Signed Ranked Test and descriptive statistics. Moreover, the qualitative data of students’ opinion and reflection were used to supplement the conclusion of the study. The experimental study result are summarized as follows: 1. Applying board games on Chinese teaching does improve student’s learning outcomes significantly. 2. Applying board games on Chinese teaching enhances parts of student’s Teaching abilities in Chinese significantly.

1 Introduction Using board games to activate teaching in the classroom could not only enhance students’ learning interests but also their interaction, social skills, and opportunities for social observation (Lavoie 2002), and through solving the problem of board game design to learn high-level thinking ability, and then achieve the effect of learning transfer and the change of behavior and attitude (Harris 2009). It should be considered their learning characteristics while designing Aboriginal children’s teaching strategy (LeMoine 2001) to design appropriate learning methods for teaching. The learning model suitable for Aboriginal children is quite different from the traditional method of narration like students listening to the audience and regularly studying in the existing framework. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 334–343, 2022. https://doi.org/10.1007/978-3-031-08819-3_35

The Impact of Integrating Board Games into Chinese Teaching

335

Therefore, teachers engaged in teaching cannot be “teaching for teaching” but must be based on “teaching for students to learn.“ It is a valuable teaching tool to make good use of multiple teaching methods, to arouse students ‘learning motivation, and to establish students’ “learning” attitude, so as to improve the learning effect. So the researchers will use the board game for students to learn by doing for group-operated methods on indigenous students to improve their learning effectiveness effectively. The purpose of this study is to explore the impact of the integration of board games into elementary school language teaching on learning effectiveness. The research objectives are as follows: 1. To explore the integration of board games into the elementary school’s Chinese teaching to improve the learning effectiveness of students. 2. To explore how to integrate of board games into elementary school’s Chinese teaching improves students’ Chinese learning performance.

2 Literature Review 2.1 Game-Based Learning Teed (2004) analyzes three characteristics of game-based teaching: 1. Learning through games can motivate learners. 2. When students are immersed in games, the effect of learning results is significant. 3. During the game, students are confident and less afraid of failure, and can achieve the sophisticated effect of repeated learning. Therefore, if teachers can make good use of board game teaching instead of narrative methods in the classroom, they will be able to attract students’ concentration and enhance their interest in learning. 2.2 Board Game Table-top game also be called unplugged games, it can also be said as a general term for card games, board games, and tile-based games. Currently, chess, chess, playing cards, Monopoly can be said to be a type of board game. Board games not only have entertainment effects and enhance interpersonal relationships, but also allow teachers to incorporate elements of education and learning in the classroom, allowing learners to brainstorm and activate creativity in games. The benefits of strengthening emotional management and improving learner stability (Ozorio and Fong 2004). Mayer and Harris (2010) also point that modern table games have the following features that make them useful for teachers in the classroom, including: 1. 2. 3. 4.

The content is informative. The player’s decision is open. Determine the winner by scoring The theme is proportionate.

336

C.-S. Torng et al.

Caldwell (1998) also agreed that board games are helpful to learners’ arithmetic and problem-solving skills. Through board games, you can improve your thinking ability, memory, and judgment. You can also cultivate and learn how to get along and communicate with others, and make the relationship between parents and children more harmonious. 2.3 Benefits of Incorporating Board Games into Teaching Caldwell (1998) pointed out that board games have obvious benefits for children’s ability to learn mathematical operations and problem solving; Cavanagh (2008) comprehensive past research shows that the use of board games in teaching has the potential benefit of enhancing children’s mathematical operation skills. Especially for school children from disadvantaged ethnic groups. The teaching object of this study belongs to the senior elementary school stage, and the learning of the course begins to come into contact with complex and highly understood sentence structure. In order to solve the problems encountered by students in learning, in addition to providing a variety of teaching strategies, teachers also need to design these strategies and methods into courses suitable for student learning. Therefore, more and more instructional designs also use various teaching strategies and rich media to arouse learning motivation, so that students can get back their enthusiasm for learning, and look forward to more diverse and interesting teaching.

3 Methods 3.1 Research Framework According to the research motivation, research purpose, and the results of literature research, a research framework is proposed. The self-variant is the teaching strategy of integrating board games into the elementary school’s national language; the dependent variable is the learning effect and learning ability of the students’ national language; the control variable is the teaching method and teaching materials. According to the first three units of the four major national language courses of the Ministry of Education, the experimental teaching will last for ten weeks. The research architecture is shown in Fig. 1:

teaching strategy

Learning effectiveness Learning performance

Teaching methods teaching materials Fig. 1. Research framework

The Impact of Integrating Board Games into Chinese Teaching

337

3.2 Research Object The research object of this study was the senior students of the elementary school in Nantou County, which is currently taught by the researcher. There were 15 students in the class, 8 boys and 7 girls. In terms of Chinese literacy skills, more than half of the students need to be strengthened. Whether in the form or meaning of the characters, most of them have typos in writing sentences or articles, and the use and mastery of idioms need to be strengthened. For reading ability, except for a few specific students, due to their own factors (not focused, lazy…), most students can master basic reading skills to retrieve information and directly use text content. Combine and explain relevant data to find the correct conclusion. For composition ability, most students can use words to express their emotions and needs, write smooth and appropriate sentences, and use appropriate punctuation. However, the teacher’s guidance is still required to complete the writing steps such as reviewing questions, ideas, selecting materials, and organizing. In addition, only a few students can use a variety of writing skills in the article, such as metaphors, personification… 3.3 Instructional Design 1. Implementation time 1 session per week (including story card production and ending card writing), implemented for 10 weeks, a total of 10 sessions. 2. Curriculum Design Concept Researchers hope to bring interactive, high-knowledge table games to the Chinese teaching site, so that each student can use the new vocabulary and text content in Chinese textbooks to publish and create reasonable stories. The way to evaluate the game from the whole class, group to individual, gradually dismantle the scaffold, so that each student can become the host in the classroom instead of the guest, and get feedback from the teacher and peers. 3. Teaching Objectives. (1) expanding words and applications through understanding, imagination and creativity (2) use the five clues of “character, item, place, state, event” to write a reasonable essay (3) ability to express themselves, communicate and share 4. Course content. (1) once upon a time board game The board games are used to attract students ‘concentration. The gameplay can create different gameplay methods according to different teaching content, so as to cultivate students’ imagination, creativity and logical thinking ability in the game. (2) teaching content The connection between this course and the past is that the students’ game tasks expand from “words” to “one sentence” and finally “story”. Therefore, before the game, students must have the preparatory ability of “words-sentences-story”. Therefore, the four major aspects of curriculum design are expand the vocabulary, shorter sentences, reasonable sentences and essay Story.

338

C.-S. Torng et al.

(3) actual course progress The course is 10 weeks long and is divided into three phases including expand vocabulary, write a reasonable short passage and sentences, combining elements from previous tabletop gameplay and adding stories. 3.4 Data Processing and Analysis After the data of all the research subjects were collected, filed, and confirmed to be correct, the data was quantified and analyzed with the SPSS25.0 statistical software package. 1. discussion on the improvement of the learning effect of the integration of board games into the elementary school’s national language teaching. Based on the 107 national elementary school remedial teaching technology assessment standards, the Wilcoxon symbol level test in the non-major test was used to compare the difference between the average scores before and after the test. Whether there is a significant difference in the learning effectiveness after teaching Chinese as a basis for improving learning effectiveness. 2. explore the improvement of the Chinese language learning performance of the elementary school students in the integration of board games. Based on the 107-year national elementary school remedy teaching science and technology assessment standard, the Wilcoxon symbol level test method in the nonmajor test was used to compare the results of the basic learning content (ability index) of the pretest and posttest to detect the acceptance of the research object. Is there a significant difference in learning performance after board games are integrated into the elementary school teaching?

4 Results and Discussion 4.1 Test of Differences in Students’ Chinese Growth Test This section will explore whether the subjects have significant differences in scores before and after the test of the Chinese language growth test for scientific and technological assessment of remedial teaching after ten weeks of experimental course teaching. The following will list the mean and standard deviation, and use the Wilcoxon symbol level test in the non-mother test to confirm the significance of the difference before and after the Chinese test. There were 15 subjects. The average score of the pre-test in the Chinese Growth Test was 63.07 points and standard deviation. Is 15.733; the average post-test score is 82.13 points, and the standard deviation is 12.467. The national language growth test for the science and technology assessment of remedial teaching in the national elementary school. The grades of grades 5 and 6 of the test questions passed the standard with a correct answer rate of 72%. The subject’s pass rate measured before May was 33.33%; the pass rate measured after December was 86.67%, and the pass rate increased by 53.34%.

The Impact of Integrating Board Games into Chinese Teaching

339

From the results of Wilcoxon’s symbol-level test in the non-maternal test, it can be seen that after differentiating the measured values of the 15 subjects before and after the test, the negative (the post-test score is lower than the pre-test score) and the positive grade (The post-test score is higher than the pre-test score). The Z-value of the difference test is -3.417, and the p-value is .001, which reaches the statistical level (p < .05). The null hypothesis should be rejected, indicating that the board game is integrated into the primary Chinese teaching has a significant effect on improving students’ learning effectiveness. 4.2 Test of Differences Between Pretest and Posttest of Students’ Basic Learning Content of Chinese According to the verification results shown in Table 1, the following situations can be observed and analyzed: (1) Sentences-Reading and Reading In the assessment of the science and technology of remedial teaching in the national elementary school, the basic learning content of “segment-reading and reading” was studied. After receiving the board game and integrating into the Mandarin teaching, one research subject’s post-test score was lower than pre-test. Grades; post-test scores of 14 subjects were higher than pre-test scores. Using Wilcoxon symbol level test, the magnitudes of the front and post-tests of 15 subjects were differentiated into grades, with negative grades (post-test scores lower than pre-test scores) and positive grades (post-test scores higher than pre-test score), the Z value of the difference test is −3.240, and the p value is .001, which reaches the statistical level (p < .05). The null hypothesis should be rejected, indicating that the board game is integrated into the elementary school language teaching. The improvement of the “Segment-Reading and Reading” learning performance has significant effects. (2) Words-Application In the science and technology assessment of national primary school remedial teaching, the basic learning content of “words and applications” was studied. After receiving board games and integrating into Mandarin teaching, three of the research subjects had lower post-test scores than pre-test scores. The post-test scores of 8 subjects were higher than the pre-test scores; the post-test scores of 4 subjects were equal to the pre-test scores. Using Wilcoxon symbol level test, the magnitudes of the front and post-tests of 15 subjects were differentiated into grades, with negative grades (post-test scores lower than pre-test scores) and positive grades (post-test scores higher than pre-test score), the Z-value of the average difference test is − 2.409, and the p-value is .016, which reaches the statistical level (p < .05). The null hypothesis should be rejected, indicating that the board game is integrated into the elementary school language teaching. The improvement of word-app learning performance has significant effects. (3) Words-Recognition In the science and technology assessment of national primary school remedial teaching, the basic learning content of “words-recognition” was studied. After

340

C.-S. Torng et al.

receiving board games and integrating into Mandarin teaching, two subjects’ posttest scores were lower than pre-test scores.; Post-test scores of 11 subjects were higher than pre-test scores; Post-test scores of 2 subjects were equal to pre-test scores. Using Wilcoxon symbol level test, the magnitudes of the front and posttests of 15 subjects were differentiated into grades, with negative grades (post-test scores lower than pre-test scores) and positive grades (post-test scores higher than pre-test score) The Z-value of the average difference test is −2.231, and the pvalue is .020, which reaches the statistical level (p < .05). The null hypothesis should be rejected, indicating that the board game is integrated into the elementary school language teaching. The improvement of the learning performance of “words-recognition” has had significant effects. (4) Chapter-Reading Aloud and Reading In the national elementary school remedial teaching technology assessment, the basic learning content of “chapter-reading aloud and reading”, after the study subjects accepted board games into the Mandarin teaching, 5 of the research subjects’ post-test scores were lower than the pre-test score; The post-test score of 9 subjects was higher than the pre-test score; the post-test score of 1 subject was equal to the pre-test score. Using Wilcoxon symbol level test, the magnitudes of the front and post-tests of 15 subjects were differentiated into grades, with negative grades (post-test scores lower than pre-test scores) and positive grades (post-test scores higher than pre-test score) Z-value of the difference test is −1.096, p-value is .273, which does not reach the statistical level (p < .05), should accept the null hypothesis, indicating that the board game is integrated into the elementary school teaching The improvement of the “Chapter-Reading and Reading” learning performance has not achieved significant results. Table 1. Pre-test and post-test Wilcoxon test integrated into the basic learning content of Chinese language teaching Learning content

Group

Sentences-read aloud and read

post-test < pre-test

1

post-test > pre-test

14

post-test = pre-test

0

post-test < pre-test

3

post-test > pre-test

8

Word-application

Word-recognition

Chapter-Reading and Reading

* p < .05 ** p < .01

Number

post-test = pre-test

4

post-test < pre-test

2

post-test > pre-test

11

post-test = pre-test

2

post-test < pre-test

5

post-test > pre-test

9

post-test = pre-test

1

Z

p

−3.240**

.001

−2.409*

.016

−2.321*

.020

−1.096

.273

The Impact of Integrating Board Games into Chinese Teaching

341

To sum up, in the statistical data, after receiving the board game and integrating into the Chinese teaching of elementary school, the basic learning content of the 107 national elementary school remedial teaching technology assessment of the Chinese language growth test is in the previous post-test difference level. Results Under the statistical test, the test scores of the study subjects in the “segment-reading and reading”, “word-application” and “word-recognition” average scores reached statistical standards, indicating that the board game is integrated into the primary language The teaching has a significant effect on the improvement of the learning performance of “segment-reading and reading”, “word-application” and “word-recognition”. The difference in the average score of the basic learning content of “Chapter-Reading and Reading” has not reached the statistical level, indicating that the integration of board games into elementary school Mandarin teaching has not significantly improved the learning performance of “Chapter-Reading and Reading”. This study shows that after the board game is integrated into the elementary school’s Mandarin teaching, the test subjects’ statistical differences in the average scores of “segment-reading and reading”, “word-application” and “word-recognition” have reached statistical significance. Level, which shows that most of the ability that students can improve belongs to the middle and low level; and the improvement of the higher level ability “text-reading and reading” learning performance has no significant effect.

5 Conclusion and Suggestion 5.1 The Integration of Board Games into Chinese Teaching in Elementary Schools has significantly Improved Student Learning Effectiveness The research objective of this study is “to explore the improvement of the learning effect of the integration of board games into the elementary school’s Mandarin teaching.“ After ten weeks of experimental course teaching, the average number of post-test scores in the Mandarin Growth Test assessed by the science and technology of remedial teaching is greater than the average of pre-test scores. The difference test of the averages has reached a statistical level. The table shows significant progress, so the research result is that--the integration of board games into the elementary school’s Mandarin teaching has a significant effect on the improvement of student learning. 5.2 The Integration of Board Games into the Teaching of Chinese in Elementary Schools has Significantly Improved the Performance of Students in Learning Mandarin The second purpose of this research is to “explore the improvement of students’ Chinese language learning ability by incorporating board games into elementary school Chinese teaching”. According to the 107-year national elementary school remedial teaching science and technology assessment, the comparison of the previous post-test basic learning content shows that, After ten weeks of experimental course teaching, the research subjects have achieved significant results in improving their learning performance in “segment-reading and reading”, “word-application” and “word-recognition”.

342

C.-S. Torng et al.

5.3 Suggestions (1) Timely integration into table games in teaching. The results of this study found that the integration of board games into the learning method of Mandarin teaching can effectively improve students’ learning effectiveness. It can also be learned from the open questionnaire that students like to use this learning method, and most of them give positive feedback. Therefore, it is recommended that teachers can use the board game learning method at the appropriate time when teaching, or adapt or design a board game that meets the learning theme of the students to use in the classroom, and also recommend that teachers choose clear patterns and colors It is rich and the content is in line with the background experience of the learner. If it is a lower grade learner, it is better to choose a game card with phonetic transcription. (2) Ability gap arrangements and interaction among group members. Board games are mostly conducted in groups in the classroom. During the board game activities, although there will be competition and victory among team members on the one hand, it also requires the cooperation of group members on the other. Complete the goals together, so when the board games are grouped, the composition of each group’s members becomes very important. We must consider the gaps in students ‘learning ability, the students’ individual differences, and their interactions with peers in the class. Teachers must not only assess whether the characteristics, content, and mechanism of the board games used are suitable for the students, but also consider the impact of the level of students’ learning. For students with high academic achievements, they may find it boring and lacking interest, so they can assign tasks to him, such as asking him to be a little helper or teacher, and even ask him to be a referee, prompter or order. (3) the psychological impact of students. Different from traditional teaching methods, the use of board games in the classroom has greatly increased the interaction between students and students. In board games, the relationship between students is competition and cooperation. There is competition and cooperation. Teachers are advised to provide whole class counseling before the game. Promoting winning or losing is not the most important part of the board game. It is to invite students to enjoy the group members The interactive process reduces the negative emotions of students due to their good intentions and affects the learning effect. Therefore, when the board game is in progress, the teacher is best to observe it and visit each small group, and then care about the students’ learning performance and emotional fluctuations. (4) Multiple evaluation methods. Teachers should make good use of effective evaluation methods to evaluate the learning effectiveness of students. In the process of teaching activities, continuous evaluation should be performed to achieve the purpose of teaching interaction, while focusing on the learning process and growth and change. Students’ participation in the classroom can also be included in one of the scoring criteria. For example, active participants or those who achieve the task can reduce part of the homework as a reward for intentionally participating in classroom activities. Teachers can help students understand their

The Impact of Integrating Board Games into Chinese Teaching

343

learning patterns and progress through interactions between students, and students can clarify ideas and gain deep understanding during learning interactions.

References Caldwell, M.L.: Parents, board games, and mathematical learning. Teach. Child. Math. 4, 365–367 (1998) Cavanagh, S.: Playing games in class helps students grasp math. Educ. Week 27, 43–46 (2008) Harris, C.: Meet the new school board: Board games are back and they”re exactly what your curriculum needs. School Lib. J. 55, 24–26 (2009) Lavoie, R.: The teacher’s role in developing social skills. Retrieved April 27 (2011). http://www. ricklavoie.com/teacherart.html LeMoine, N.R.: Language variation and literacy acquisition in African American students. In: Garris, J.L., Kamhi, A.G., Pollock, K.E. (eds.) Literacy in African American communities, pp. 169–194. Hampton Press, Mahwah (2001) Mayer, B., Harris, C.: Libraries Got Game: Aligned Teaching Through Modern Board Games. American Library Association, Chicago (2010) Ozorio, B., Fong, D.K.C.: Mandarin casino gambling behaviors: Risk taking in casinos vs. investments. UNLV Gaming Res. Rev. J. 8(2), 27–38 (2004) Teed, R.: Starting Point-Teaching Entry Level Geoscience [Game-Based Learning] (2004). http:// serc.carleton.edu/introgeo/games/index.html

Impacts of COVID-19 on Stock Returns of the Cross-border Transportation Industry Ying-Li Lin1 , Kuei-Yuan Wang1(B) , and Ching-Ru Yang2 1 Department of Finance, Asia University, No. 500, Liufeng Rd., Wufeng Dist.,

Taichung City 413305, Taiwan [email protected], [email protected] 2 Dong Ming Elementary School, No. 93, Dongming Rd., Dajia Dist., Taichung City 437106, Taiwan

Abstract. This research examined the influence of COVID-19 on the stock returns of the cross-border transportation industry by performing an event study on the cumulative abnormal return of cross-border transportation companies listed in Taiwan from December 2019 to February 2021. The purpose is to analyze whether the pandemic has a negative impact on the stock returns of the cross-border transportation industry. The empirical results indicate the cumulative abnormal return is indeed influenced by COVID-19, but the impact can be positive or negative at different time points.

1 Introduction The COVID-19 pandemic spread around the world in 2020. International cruise ships were not even allowed to dock at harbors in Taiwan. Taiwanese business people returning from China had to take designated flights, while the public shunned away from marine and air transportation due to the higher likelihood of infections. Data from Anue1 (a financial website) suggest that Evergreen Marine Corporation posted annual revenue of NT$181.275 billion in 2019 and then NT$89.049 billion in 2020. Its share price dropped from NT$13 to the lowest point of NT$8. Data for the first quarter of 2020 show that earnings per share turned from positive to negative for two consecutive quarters. Yang Ming Marine Transport generated a revenue of NT$149.257 billion in 2019 and NT$151.884 billion in 2020. Its earnings per share turned positive in the third quarter of 2020 after losses for six consecutive quarters. Given the significant impact of COVID-19 on cross-border transportation companies from Taiwan, this research conducted an event study to explore the pandemic effect on the stock returns of cross-border transportation companies in Taiwan. The research objectives are as follows. (1) To examine whether COVID-19 caused significant and abnormal stock returns of TWSE-/TPEx-listed cross-border transportation companies in Taiwan. (2) To explore the abnormal returns of the air and marine transportation industry in the context of the semi-strong form of the efficient market hypothesis if the tests indicate statistical significance. 1 Anue website at https://stock.cnyes.com/market/TWS:2618:STOCK/eps.

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 344–350, 2022. https://doi.org/10.1007/978-3-031-08819-3_36

Impacts of COVID-19 on Stock Returns

345

2 COVID-19 Impact on the Cross-border Transportation Industry According to data from the Ministry of Health and Welfare2 , COVID-19 has been spreading around the world since December 2019. Beginning in January 2020, Taiwan implemented a series of measures on airline passengers, such as a ban on package tours to China and restrictions on travelers from China. Starting on February 2, 2020, a 14-day home quarantine was required for those coming from mainland China to Taiwan via the mini three links. Asymptomatic individuals from China, Hong Kong, and Macao had to complete self-health management. Statistics from the Civil Aeronautics Administration indicate that the inbound load factor of international airlines and the outbound load factor declined by 62.4% and 55.6% in 2020 from 2019, respectively. The average load factor during the first seven months of 2020 dropped to 59.8% from 82.4% year-over-year. However, some airlines identified opportunities amid the sweeping effects of the pandemic. Air transportation is the most efficient way of transporting materials and resources for COVID responses around the world. Thus, many cross-border transportation companies switched gears by transporting anti-COVID supplies or other essentials in order to create economic value and maintain competitiveness.

3 Research Methods 3.1 Research Period and Data Sources To ensure data integrity and accessibility, this study sampled cross-border transportation companies listed on the Taiwan Stock Exchange (TSEC) that have comprehensive financial, ownership structure, and share price data. The research period spanned from December 2019 to February 2021. The daily return data over this period of more than two years were sourced from Taiwan Economic Journal. In total 14 companies were sampled (Table 1). Table 1. Sampled companies China Ship Building Corporation (2208)

CMT Logistics Co., Ltd. (2612)

Evergreen Marine Corp. (2603)

Wan Hai Lines, Ltd. (2615)

Sincere Navigation Corporation (2605)

Taiwan Navigation Co., Ltd. (2617)

U-Ming Marine Transport Corporation (2606) Eva Airways Corporation (2618) Evergreen International Storage & Transport (2607)

T.H.I. Logistics Co., Ltd. (2636)

Yang Ming Marine Transport Corporation (2609)

Farglory F T Z Investment Holding Co., Ltd. (5607)

China Airlines Ltd. (2610)

Shih Wei Co., Ltd. (5608)

2 Ministry of Health and Welfare, https://www.mohw.gov.tw/mp-1.html.

346

Y.-L. Lin et al.

3.2 Empirical Model Event studies are often used for empirical research in finance and the economy (Ball and Brown 1968; Fama et al. 1969). This method allows one to examine whether a specific event creates abnormal returns (AR) on stock prices and to conduct tests on whether abnormal returns are significantly different from zero. This research performed an event study in the following four steps. (1) Definition of event days Table 2 summarizes the event days when the Ministry of Health and Welfare announced material information. Table 2. COVID-19 event days in the cross-border transportation industry Event code (MM/DD/YYYY) Event E1

January 22, 2020

First confirmed case in Taiwan on January 21, 2020. The event day defined by this study is January 22, 2020 when COVID-19 had an impact on the cross-border transportation industry

E2

April 19, 2020

First infections and a total of 36 confirmed cases in Goodwill Fleet of the R.O.C. Navy on April 18, 2020. The event day defined by this study is April 19, 2020 when COVID-19 had an impact on the cross-border transportation industry

E3

January 13, 2021

First infections and a total of 19 confirmed cases at Taoyuan General Hospital, Ministry of Health and Welfare on January 12, 2021. The event day defined by this study is January 13, 2021 when COVID-19 had an impact on the cross-border transportation industry

(2) Definition and estimation of abnormal returns. (3) Tests on abnormal returns. (4) Analysis and interpretation of empirical results.

4 Empirical Results (1) Test on cumulative average abnormal returns (CAAR) during Event Day E1 According to the test on CAAR shown in Table 3, the findings are statistically insignificant before the event day. The CAAR after the event day were mostly negative and extremely significant.

Impacts of COVID-19 on Stock Returns

347

Table 3. Empirical results. CAAR

T-statistic

P-Value

0.28%

0.8491

0.3958

(−10, −9)

−0.44%

−0.9234

0.3558

(−10, −8)

−0.35%

−0.6021

0.5471

(−10, −7)

−0.32%

−0.4732

0.6361

(−10, −6)

−0.60%

−0.8039

0.4215

(−10, −5)

−1.03%

−1.2679

0.2048

(−10, −4)

−0.92%

−1.0482

0.2946

(−10, −3)

−0.51%

−0.5359

0.5921

(−10, −2)

−0.76%

−0.7579

0.4485

(−10, −1)

−0.76%

−0.7211

0.4708

(−10, 0)

−5.39%

−4.8777***

0***

(−10, 1)

−5.33%

−4.6145***

0***

(−10, 2)

−8.53%

−7.0972***

0***

(−10, 3)

−8.30%

−6.6562***

0***

(−10, 4)

−8.67%

−6.7149***

0***

(−10, 5)

−7.24%

−5.4354***

0***

(−10, 6)

−8.03%

−5.8451***

0***

(−10, 7)

−8.92%

−6.3117***

0***

(−10, 8)

−8.30%

−5.7115***

0***

(−10, 9)

−7.96%

−5.3442***

0***

(−10, 10)

−7.55%

−4.9458***

0***

(−10, −10)

Note: ***, **, * Denote coefficient estimates that are reliably significant at the 1%, 5%, 10% levels, respectively

(2) Test on cumulative average abnormal returns (CAAR) during Event Day E2. The cumulative average abnormal returns (CAAR) on Event Day E2 are presented in Table 4. They started in negative territory on t = −10 and gradually climbed to positive territory on t = 10. The highest point was on t = 7, (CAR = 7.82%, t = 4.749, p < 0.01). The range from t = -2 to t = 0 exhibits an obvious decline. This was followed by an immediate rebound. In other words, the impact on the cross-border transportation industry in Taiwan was relatively short-lived in this stage. (3) Test on cumulative average abnormal returns (CAAR) during Event Day E3.

348

Y.-L. Lin et al. Table 4. Empirical results. CAAR

T-statistic

P-Value

(−10, −10)

−1.32%

−3.4125***

0.0006***

(−10, −9)

−1.84%

−3.3605***

0.0008***

(−10, −8)

−0.42%

−0.6259

0.5314

(−10,−7)

2.48%

3.1962***

0.0014***

(−10,−6)

3.21%

3.7052***

0.0002***

(−10, −5)

2.99%

3.1481***

0.0016***

(−10, −4)

2.33%

2.2722**

0.0231**

(−10, −3)

3.66%

3.3371***

0.0008***

(−10, −2)

3.68%

3.1653***

0.0015***

(−10, −1)

2.01%

1.6374

0.1016

(−10, 0)

2.22%

1.7232*

0.0849*

(−10, 1)

4.00%

2.976***

0.0029***

(−10, 2)

3.62%

2.5858***

0.0097***

(−10, 3)

4.05%

2.7889***

0.0053***

(−10, 4)

3.78%

2.5168**

0.0118**

(−10, 5)

3.95%

2.5428**

0.011**

(−10, 6)

5.86%

3.6663***

0.0002***

(−10, 7)

7.82%

4.7494***

0***

(−10, 8)

7.28%

4.3048***

0***

(−10, 9)

7.29%

4.2016***

0***

(−10, 10)

7.43%

4.1785***

0***

Note: ***, **, * Denote coefficient estimates that are reliably significant at the 1%, 5%, 10% levels, respectively

Table 5 shows the gradual cumulation of abnormal returns starting from t = -10. The cumulative abnormal returns reached the lowest point in the negative territory on t = 6. The returns are mostly negative and extremely obvious.

Impacts of COVID-19 on Stock Returns

349

Table 5. Empirical results. CAAR

T-statistic

P-Value

(−10, −10)

−0.87%

−1.3339

0.1823

(−10, −9)

−3.72%

−4.0157***

0.0001***

(−10, −8)

−2.70%

−2.382**

0.0172**

(−10, −7)

−1.47%

−1.1237

0.2611

(−10, −6)

−3.68%

−2.5142**

0.0119**

(−10, −5)

−8.59%

−5.3562***

0***

(−10, −4)

−13.98%

−8.0687***

0***

(−10, −3)

−13.06%

−7.0478***

0***

(−10, −2)

−10.88%

−5.5367***

0***

(−10, −1)

−15.63%

−7.5447***

0***

(−10, 0)

−17.88%

−8.2313***

0***

(−10, 1)

−15.96%

−7.0337***

0***

(−10, 2)

−21.95%

−9.295***

0***

(−10, 3)

−22.20%

−9.058***

0***

(−10, 4)

−25.84%

−10.1888***

0***

(−10, 5)

−30.11%

−11.4953***

0***

(−10, 6)

−30.76%

−11.3903***

0***

(−10, 7)

−27.10%

−9.7519***

0***

(−10, 8)

−26.31%

−9.2162***

0***

(−10, 9)

−27.02%

−9.2253***

0***

(−10, 10)

−27.82%

−9.2695***

0***

Note: ***, **, * Denote coefficient estimates that are reliably significant at the 1%, 5%, 10% levels, respectively

5 Conclusion This research examined the impact of COVID-19 on the volatility of the stock prices of cross-border transportation companies in Taiwan, using the event study module provided by Taiwan Economic Journal. The event period is defined as ten days before and after the event. The estimation period for the calculation of cumulative abnormal returns covered a total of 150 days. The empirical results indicate that the first confirmed case in 2020 caused negative abnormal returns and the impact continued after the event day. In other words, the event had a material influence on stock prices. The cluster infection in early 2021 also led to negative abnormal returns. The abnormal returns dropped on the event day and reached an extremely significant level. After the event day, the abnormal returns were sometimes positive and other times positive.

350

Y.-L. Lin et al.

In response to COVID-19, Taiwan implemented certain restrictions on inbound and outbound travelers. This indeed influenced airliners and relevant marine transportation companies. At the same time, global trade was affected, and the supply of ships and containers declined dramatically. The situation turned into a growth opportunity for cross-border transportation companies in Taiwan. Future researchers can examine the effect of other events (e.g., blocking of the Suez Canal by Ever Given of Evergreen Marine Corporation and the travel bubble plan) during the time period. This will make the research strand wider and more complete. Acknowledgments. This research was supported by Ministry of Science and Technology of the Republic of China under contract MOST 110-2813-C-468-083-H.

References Ball, R., Brown, P.: An empirical evaluation of accounting income numbers. J. Account. Res. Autumn, 159–178 (1968) Fama, E.F., Fisher, L., Jensen, M.C., Roll, R.: The adjustment of stock prices to new information. Int. Econ. Rev. 10, 1–21 (1969)

Study on Business Continuity of Small and Medium-Sized Firms in Japan: Focusing on Business Continuity Planning for Natural Disaster Risk Mei Hua Liao1 and Hidekazu Sone2(B) 1 Asia University, Taichung City, Taiwan

[email protected]

2 Shizuoka University Art and Culture, Hamamatsu, Japan

[email protected]

Abstract. This paper discusses BCPs for natural disaster risks such as earthquakes and tsunamis as a study on business continuity of Japanese small and medium enterprises (SMEs).Before discussing BCPs, a review of previous studies on risk management is conducted, followed by a case study of Suzuyo Corporation, a company in Shizuoka Prefecture bordering the Pacific Ocean, against natural disaster risks. A case study of a company in Shizuoka Prefecture bordering the Pacific Ocean is presented. Through this study, we hope to discuss the efforts of a cutting-edge company in Japan, a country that is prone to natural disasters, and hope that you will take an interest in this topic.1 Introduction.

1 Introduction This paper discusses the evolution of risk management, which has been the focus of much attention in recent years and discusses measures that companies are taking to cope with natural disasters. Although the concept of risk and preparedness has existed since ancient times, in the modern era, the birth of corporations, the expansion of the economy and the scope of their activities, and their increasing influence have led to a focus on actions and events related to risk. Since then, whenever a crisis occurred in the survival of a nation or a company, such as two world wars, the Great Depression, oil shocks, and nuclear accidents, risk has been deeply considered, and research has progressed. This led to the study of various risks, including social risks, and the term “risk” has been used frequently since then to refer to social recessions, corporate scandals, mergers and acquisitions, earthquakes and other natural disasters. Furthermore, the risks that companies face are increasing and becoming more complex every year, such as internationalization, M&A, compliance issues, and information leaks. For this reason, not a day goes by without seeing the word “risk” in the daily newspapers, and special issues are published in economic journals and academic circles. However, there have been various interpretations of the concept and system. In this paper, we will discuss the basic items and present some issues and future directions. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 351–359, 2022. https://doi.org/10.1007/978-3-031-08819-3_37

352

M. H. Liao and H. Sone

In recent years, natural disasters such as earthquakes and tsunamis have hit Japan, causing tremendous damage. Among them is the Nankai Trough earthquake, which is expected to cause significant damage within the next few years. The case study shows how companies in Shizuoka Prefecture, which faces the Pacific Ocean and is threatened by tsunamis, are responding to these risks, and how they are implementing innovative measures through collaboration with local governments, consultants, and researchers. Field surveys of these companies will be conducted and case studies will be presented.

2 Concept of Risk Before discussing risk management, which is also the theme of this paper, I would like to discuss what risk is in the first place. According to Bernstein [1], the origin of the word “risk” can be traced back to the Italian word risicare, which means “to try courageously. On the other hand, according to Ishii [2], the English word “risk” originally comes from the Latin word risicare. The word itself has the meaning of “navigating between rocks. Just as there are various theories on the etymology of the word, the definition of risk is similar, and since it is discussed from various perspectives, there is no fixed definition. 2.1 Risk from an Economics Perspective People’s perception of risk has been sensible since before the birth of economics. According to Tachibanaki et al. [3], life insurance systems have existed since Greek and Roman times, and people were prepared for the risk of death. From an economics perspective, Bernoulli, a mathematician, and Adam Smith, the “father of economics,” were pioneers in the field. It was a landmark paper that discussed the effectiveness of the “expected utility criterion” as a decision-making criterion under risk. However, this paper was too far ahead of its time and was ignored for more than 200 years (Sakai [4]). Adam Smith, on the other hand, focused on the fact that lotteries had been successful in spreading the lottery, and he also made an in-depth study of risk. Later, economists such as Laplace, Marshall, Willet, Knight, and Keynes developed the relationship between risk and uncertainty (Knight [5]) and related insurance, and eventually laid the groundwork for risk management theory. In economics, a distinction is also made by calling the difficulty of assigning probabilities to such events “uncertainty” (Sakai [6]). In other words, fluctuations whose probability can be calculated and measured are considered “risk,” while fluctuations whose probability is difficult to calculate and measure are simply “uncertainty. Sakai [6], an economist who has studied risk economics for many years, defines risk as follows. Risk means the degree of uncertainty of an “undesirable event” and the magnitude of its consequences for the maintenance of human life and socioeconomic activities. A large risk means that the probability of the occurrence of an undesirable event is large and the extent of its damage is large” (p.13), including the relationship with uncertainty. As pointed out by many related researchers as well as Sakai, the term “risk” itself is originally a neutral term. However, it is often seen as a negative aspect of risk. Since risk plays a major role in human activities themselves, it has recently been questioned

Study on Business Continuity of Small and Medium-Sized Firms in Japan

353

how to make better use of risk and lead it in a better direction, instead of regarding risk only as a negative aspect. Many recent studies have discussed “risk” as a concept that includes both negative aspects. Sakai [6], while referring to the phrase “risk is a drug, and a drug is a risk,” states, “It is not good to view risk only in terms of ‘negative aspects,’ such as disasters or moral degradation. A moderate level of risk can also be a “drug” that stimulates human activity. It is important to pay attention to the positive aspects of risk” (p. 61). He points out the importance of cultivating a correct view of the risk society and paying close attention to avoid misunderstanding the word “risk” itself. 2.2 Risk from the Perspective of Business Administration In recent years, risk has been attracting increasing attention in the field of business administration. Similarly, management scholar Kagono [7] argues that “as has been said for a long time, the greatest risk is that people stop challenging risk” (p. 9), and points out that corporate management and risk have a very important relationship. Simons [8] also points out that risk is not a bad thing and that challenging risk is a source of innovation and creativity. In the field of business administration, the general perception of risk in the past was that risk was something to be feared and that negative factors were at the forefront. However, in addition to the above-mentioned points, recent years have seen a number of studies discussing how firms should deal with risk, especially from the perspective of business administration and management strategy (Okumura, [9] etc.). 2.3 Classification of Risks There are countless risks surrounding us. Therefore, it is a very difficult task to classify risks in detail. However, as has been discussed in previous studies, we can broadly divide risks into two categories. The first is pure risk and speculative risk. Pure risks refer to risks that cause damages only, such as natural disasters (e.g., typhoons, earthquakes, and tsunamis), fires, traffic accidents, kidnappings, and terrorist attacks. Pure risks are very difficult to predict because they occur unexpectedly and accidentally. However, if a large amount of data on such cases is observed and analyzed over a wide area and a certain period of time, it is possible to estimate the frequency of occurrence from a statistical viewpoint. Speculative risk, on the other hand, refers to the risk that one party will profit and the other party will be harmed by exchange rate fluctuations, interest rate fluctuations, stock price fluctuations, and the like. Such speculative risk is directly related to the behavior of people and the activities of nations and corporations. Second, there are static (passive) and dynamic (active) risks. Static risk is the risk of accidental exposure from outside, such as natural disasters, fires, and traffic accidents, while dynamic risk is the risk that arises when a company changes its current situation, such as by expanding into new businesses, entering overseas markets, and developing new products. Most of the subjects of risk management discussed below have so far focused on pure risks and static (passive) risks mentioned above. However, risk management related

354

M. H. Liao and H. Sone

to corporate decision-making and management strategy, which is also the objective of this paper, naturally refers to the latter dynamic (active) risks.

3 Expansion of Risk Management Risk management research has increased in many fields (in a broad sense), although many studies are limited in scope (in a narrow sense) in relation to insurance theory, which was the original source of risk management research. Later, this field has expanded with the development of research and has been discussed from economics, safety engineering, management, informatics, environmental, and disaster prevention and mitigation perspectives as well. Furthermore, the scope of risk management by field is also diverse, including failure risk, natural disaster risk, SME risk management, family risk management, food product risk management, reputational risk management, and business succession risk management. In this paper, we focus on the crisis roulette of Mitroff & Alpaslan [10] of the University of Southern California, also known as a leading risk research university. They describe a roulette of eight major risks that can occur in the course of sustained corporate management. First, normal accidents include “personnel crises,” “physical crises,” and “economic crises. Next, the abnormal risks are “criminal risks,” “information-related risks,” and “image-related risks. And finally, natural disasters.

4 Significance and Limitations In this section, we point out the significance and limitations of the preceding risk management studies, which can be broadly divided into two categories. First, as represented by the early studies of Knight [5], risk management in Europe, the U.S., and Japan has been influenced by a series of subsequent studies that discussed risk management (insurance, warranty, and contractual indemnity) (e.g., Mehr and Hedges [11]), the discussion began with health management as a part of management rationalization and cost control. For this reason, even today, the discussion focuses on studies related to such insurance management. As a result, the accumulation of research on risks other than insurance management is still shallow. In other words, research has been limited to research fields. Second, since the 1990s, risk management theory has been studied in earnest, taking up specific companies as well. However, most of the studies have focused on risk management in modern corporate activities and have explained the importance of risk management. In particular, there are almost no studies that focus on the keyword “corporate survival,” which is the subject of this paper. For this reason, we focus on firms that are also called “long-established firms” and consider risk management to be an essential part of corporate management, including the wisdom of survival over many generations, the appointment of human resources, and changes in management. We believe that risk management research, and by extension, related research on long-established firms, will not become more mature without considering these matters. Therefore, this paper attempts to provide a new perspective on risk management research.

Study on Business Continuity of Small and Medium-Sized Firms in Japan

355

5 Case Study: Natural Disaster Risk Countermeasures by Japanese Company This chapter discusses and examines the natural disaster risks surrounding Suzuyo Corporation, a major general logistics company, and its countermeasures based on a survey conducted by the company through a case study. 5.1 Suzuyo Corporation Suzuyo Co., Ltd. is one of the leading companies in Shizuoka, with its headquarters in Shizuoka City. The company was founded in 1801 by Suzuki Yohei I as Harimaya, a logistics company that used ships to distribute goods at Shimizu Minato in Suruga Province. Since its establishment, the company has been located along the Pacific coastline, which means that the company has always been in close proximity to natural disasters. The company’s documents also show how it was affected by the Great Kanto Earthquake, indicating that the company has been dealing with natural disasters for a longer period of time than other companies. Today, Suzuyo is known as a company that has diversified into a wide range of businesses, including commercial distribution, construction and building maintenance, and aviation, centered on Suzuyo’s comprehensive logistics business. The company has about 150 affiliated companies in Japan and abroad, employs 1,100 people, and had sales of 144.2 billion yen (as of August 2021). The company’s headquarters is located right in front of the Port of Shimizu, which exposes the company to the threat of not only earthquakes but also tsunamis. This prompted the company to take earthquake countermeasures after the Great Kanto Earthquake, and in recent years, it has taken its own countermeasures. An interview survey was conducted, with assistance from Daisuke Goto, General Manager, Crisis Management Office, Suzuyo Corporation (October 15, 2018), and others. 5.2 Establishment of the Crisis Management Committee In March 2005, based on the “Business Continuity Formulation Guidelines” released by the Ministry of Economy, Trade and Industry, disaster prevention measures were reviewed, the “Business Continuity Project” was launched, and the Crisis Management Office was created. The entire group took this as a group-wide initiative and implemented a BCP (Business Continuity Plan) for the entire group, assuming a huge earthquake and tsunami. The head office president is the head of the committee, which is chaired by the vice president and divided into 15 business groups (affiliated companies). When the committee was first established, it met once a month. In recent years, when the committee has gotten off the ground, it has met once every two months, which has contributed to the unity of the affiliated companies and the vitality of the organization. Before the BCP initiative, the General Affairs Department had prepared disaster supplies, but there was no generator and only a small amount of food and water. Through

356

M. H. Liao and H. Sone

the BCP initiative, we identified the minimum necessary items and made preparations. The establishment of the Crisis Management Committee led to the concept of group-wide response. 5.3 Suzuyo Group Crisis Management Committee Basic Policy There are three basic policies for crisis management. First, the first priority is to save lives, second, to maintain a relationship of trust with stakeholders, and third, to strive for mutual aid with the local community. The third is to strive for mutual aid with the local community. Particularly distinctive is the third point, mutual aid with the local community. The third characteristic is mutual assistance with the local community. The Shimizu Port BCP is being developed in cooperation with the prefectural government’s Port Management Bureau and the Ministry of Land, Infrastructure, Transport and Tourism. The Port of Shimizu is working on a plan to be ready to ship cargo on the fourth day after a disaster. Another thing is that we are not headquartered in Tokyo, Osaka, Nagoya, or any other major city, but in a place like this, so we have close ties with the local community. Mr. Goto mentioned above the importance of being rooted in the community and being closely involved with the local community since the company was founded. 5.4 Countermeasures for Natural Disaster Risk One of the company’s unique features is that its head office is located near the sea, and the company has taken measures to prepare for the risk of tsunamis. The company has diversified into many businesses, and because they are spread out over a number of regions, it is difficult to limit the risks to a specific region. The company identifies risks by asking each location to list the risks in each region, in other words, the things that would be troublesome if they were to happen now. For example, flooding is a problem, employees cannot go to work, etc. We are considering how to deal with these risks and what specific items we need to prepare to deal with them. In addition, we are also considering how to collect information and the actions of the task force in concrete terms, as well as establishing a certain number of rules as necessary. In other words, we identify the risks for each event, and based on the 4th damage assumption issued by Shizuoka Prefecture, we make a damage forecast for the location where our base is located, and combine it with the risks to make a damage forecast. Concrete preparations, coping systems, and methods were considered at the same time. The source of these ideas came from Goto’s previous experience with the Maritime Self-Defense Force. The Maritime Self-Defense Force operates by ship, so when you are in command there, there is no one to consult with, and you have to gather all kinds of information and make decisions on the spur of the moment in such situations, even if it takes a few minutes to do so yourself. That is the standard. So, although the BCP flow is written in general terms, I do most of the work based on my own ideas, so I do things that, like the risk analysis mentioned earlier, are often said by those who specialize in this field that it

Study on Business Continuity of Small and Medium-Sized Firms in Japan

357

is an evil way of doing things. I am also doing things that people who specialize in this field would often say are ill-advised. However, we are careful to do things in a way that even employees who do not normally think about these things can understand. 5.5 Equipment The company and its group companies also focus on stockpiling. All group companies have a three-day supply of food and water for employees in case of emergency. The company is also equipped with diesel generators, propane gas, solar power, and other types of fuel cells to ensure power supply. We are also making small efforts. For example, personal computers are always turned on so that they can be used at any time. Even if a power supply is plugged into an outlet, the source of the power source is not known, so the power source is indicated on the display to avoid confusion. We are also focusing on wireless equipment. We have prepared MCA, PHS, and IP radios from satellite phones and put them in company cars and executive cars so that they will not be damaged by shaking. A backup telephone switchboard has also been installed on the third floor as a backup for disaster backup telephones in case of a tsunami. 5.6 Preventing Rut The company has conducted drills on a regular basis, but has devised a variety of ways to avoid getting stuck in a rut of doing the same thing over and over again. One simple example is video screenings and quizzes for disaster prevention education. Slides are projected on the screen, and the participants are asked who they think is right in this situation, A or B. The person who is wrong is asked to sit down. Another is to encourage local residents in the neighborhood to participate in evacuation drills, so that they can stimulate each other. Rather than just practicing for the sake of practicing, we have created a model that illustrates evacuation routes and distributed it to all business locations to ensure that all employees know the evacuation routes, what to bring, and the division of roles. All bases have also created a model that allows anyone to take the place of the person in charge in the event that he or she is unavailable. In addition, as part of actual drills, the staff is made aware of the actions to be taken after an evacuation. A group task force chart has been created, and task force personnel from the information, resource management, general affairs, public relations, and other departments have gathered to form the task force. Instead of assigning a firefighting team within the task force, as is the case at other companies, we have created a separate organization that can respond flexibly, such as dispatching only the head of the emergency response team from the task force to the evacuation site, appointing people to confirm the safety of the evacuees, and deciding who will come first. If anyone is injured, you and you will take care of it. If someone is injured, you and you will take care of him or her, and you will cook the meals. The task force personnel are told that they only need to take care of this part. The idea is that employees who are not part of the task force and who have evacuated should help each other and confirm the safety of each other, including their own lives. (Daisuke Goto). However, those that need to be decided are the firefighting team, rescue operation team, and first-aid team. The rescue team, especially for stopping bleeding, gives priority

358

M. H. Liao and H. Sone

to those who raise their hands to be a member of the rescue team in the disaster prevention organization. They provide first aid, referring to the Tokyo Disaster Prevention Guidelines. That is also the SDF’s approach. When the SDF deploys people somewhere, there are certain passages in the ship that they are allowed to pass through and others that they are not allowed to pass through. There are some dangerous and some not dangerous passages, and when they are sent to the park from the back of the ship to the front, they must be instructed which passages to go through and where to go. Then, if you put it in your mind that the fire extinguishers are located here, you will probably be able to see them. (Daisuke Goto). The company also conducts a safety confirmation drill once every two months. The response rate was 98%. The remaining 2% is being addressed as a problem. What is important in safety confirmation is to identify those whose safety has not been confirmed, and to understand how many people have not been confirmed. By allowing people to freely write comments such as “there is something wrong” or “a family member was injured,” some people will add their comments, but until now, no one has confirmed their safety. However, the important thing is that since there is a possibility that there are entries such as who was injured, the room staff is instructed to check all the entries that indicate that there are abnormalities when they actually check the safety of the staff. By doing this frequently, the company aims to get the staff accustomed to replying and to establish a habit of doing so.

6 Conclusion In this paper, we have discussed the evolution of risk management and its types, and introduced some concrete examples of companies that are at the forefront of risk management, focusing on natural disaster risk, which has been the focus of much attention in recent years. Such research should be conducted not only in the field of business administration, but also in cooperation with the fields of seismology and disaster prevention. Although natural disaster risk is rarely addressed in business administration, it is necessary to pay renewed attention to it in the context of business continuity in Japan, where natural disasters are common. Lastly, I would like to present a challenge. As mentioned earlier, there have been few previous studies focusing on corporate behavior during disasters, making comparisons difficult. We believe that research into the unique response measures taken by individual companies will provide a good opportunity to consider better response measures and to share information. In addition, it will be necessary to consider not only individual companies, but also closer cooperation with the government, local communities, and other companies. Acknowledgments. This work was supported by JSPS KAKENHI Grant Numbers 18K01760, 21H00742.

References 1. Bernstein, P.L.: Against The Gods, The Remarkable Story of Risk (1996). John

Study on Business Continuity of Small and Medium-Sized Firms in Japan 2. 3. 4. 5. 6. 7. 8. 9. 10. 11.

359

Wiley&Sons,Inc. Ishii, I: Risk no shikumi (2002): Toyo Keizai Inc. Tachibanaki, T., Hasebe, K., Imada, T., Masuda, S.: Riskgaku toha nanika (2007) Shoten, I., Sakai, Y.: Risk shakai wo miru me (2006) Knight, F.H.: Risk, Uncertainty, and Profit (1921): Hart, Schaffner & Marx; Houghton Mifflin Co. 6. Sakai, Y: Risk no Keizaishisou, (2010):Mineruva Shobou Kagono, T: Deflation no shotai ha hidaisuru naiburyuhoniari, PRESIDENT, Mar, 5th (2012), pp.8–9 Simons, R.: How Risky is Your Company? Harv. Bus. Rev. 77(1), 85–94 (1999) Okumura, A: Keiei Senryaku (1989). Nihon Keizai Shimbun Mitroff, I.: Aspaslan, preparing for evil. Harv. Bus. Rev. 81(4), 109–115 (2003) Mehr, R., Hedges, B.: Risk management in the business enterprise, (1963). Homewood, III., R.D. Irwin

Comparing Investor Sentiment Between Growth and Value Stocks Mei-Hua Liao1 , Yen-Ju Chen1 , Chiung-Wen Yu1 , and Ya-Lan Chan2(B) 1 Department of Finance, Asia University, Taichung, Taiwan, Republic of China

[email protected]

2 Department of Business Administration, Asia University, Taichung, Taiwan, Republic of China

[email protected]

Abstract. Much of the behavioral finance-related literature has focused on the relationship between investor sentiment and stock returns. This study applies investor sentiment to portfolio theory. Many investors will choose to invest in value stocks or growth stocks according to their personal preferences or in different situations. This study aims to explore the relationship between value and growth stocks and investor sentiment, and establishes the following hypotheses about investor sentiment (1) Investor sentiment in growth stocks is high relative to value stocks as the market price rises; (2) Investor sentiment in value stocks is stable relative to growth stocks as the market price falls; (3) Investor sentiment in previous issue growth stocks has a positive impact on current excess returns; (4) Investor sentiment in previous issue value stocks has a positive impact on current excess returns. This study will take advantage of investor sentiment in behavioral finance to delve deeper into important issues in portfolio theory.

1 Introduction Classical finance assumes that investors are rational, but it is generally believed that people make different decisions based on emotions and personal traits in the fields of psychology. Behavioral finance discusses the contradictory phenomenon between actual financial markets and traditional finance through psychological theories. Behavioral finance treats people as an emotional individuals make irrational decisions due to various factors. A lot of literatures pointing out that irrational actions of investors affect pricing in financial markets and cause fluctuations in rewards. Kahneman and Tversky point out prospect theory, investors decisions are not in the same when encountering risks and uncertainties situation, when gaining profits, investors’ risk tolerance was risk aversion; when making a loss, investors’ risk tolerance was risk seeking [1]. The study is a major discovery of behavioral finance which explains why the risk perception of investors was inconsistent in different situations. De Bondt and Thaler found that investors have overreacting behaviors in the market [2]. When the stocks owned by investor more, profit would be more optimistic and more insensitive to bad news and be more sensitive to good news. If the holdings of stocks decrease, investors would be more pessimistic, the loser would be insensitive to © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 360–365, 2022. https://doi.org/10.1007/978-3-031-08819-3_38

Comparing Investor Sentiment Between Growth and Value Stocks

361

good news, more sensitive to bad news. Finally, the market price of the stock and the theoretical price deviation. Shefrin and Satman first declare the term, disposition effect [3]. The study found that winners in financial markets often sell stocks when they begin to earn a profit prematurely. And the losers held losing stocks too long and unwilling to stop loss. The theory is an important direction of behavioral finance study. Afterwards, many scholars conducted study on disposition effect. For instance, Weber and Camerer found that the disposition effect is real by observing the subjects [4]. However, if the stocks are automatically sold at the end of each period, the disposition effect would effectively reduce. The study of Odean also confirmed disposition effect [5]. Noise traders may try to follow decisions of investors with more information, ignoring fundamental information [6]. And noise traders are prone to make irrational investment decisions cause unpredictable risks increases in the market [7]. Then arbitrageurs exit the market to risk averse. Ultimately, the difference between market prices and theory widens. Although there have been many investors sentiment studies in Taiwan market. Investor sentiment studies on value stocks and growth stocks are relatively rare. Value and growth stocks are important indicators for many investors to choose investment portfolios. Hence, the study explores the relationship between value and growth stocks and investor sentiment in Taiwan market.

2 Literature Review Traditional finance assumes that markets work efficiently, and investors are rational. However, the actual capital market often has phenomena that cannot be explained by traditional finance. Since the end of the 20th century, some scholars have confirmed that there are situations in the market that are different from the efficient market theory. Therefore, behavioral finance has been paid more and more attention. Investor sentiment is one of the focuses of behavioral finance research. Scholars use various data on the market as indicators of investor sentiment to study various things in the financial market. 2.1 Investor Sentiment Investor sentiment refers to investors’ subjective views and expectations on the financial market, and cause investors to make irrational investment decisions, there are many ways to measure investor sentiment. Brown and Cliff divided sentiment indicators into two categories, the first is direct sentiment indicators, such as institutional surveys of investors’ views on the market. The second is indirect sentiment indicators, such as trading volume, bond ratio and IPO ratio, which can indirectly measure changes in investor sentiment [8]. Others use the weather as an indicator, such as Hirshleifer and Shumway found a significant correlation between sunlight and stock returns [9]. Huang et al. used net buy/sell of institutional investors, securities lending/borrowing rate, trading volume, the open interest of major institutional investors and put/call ratio as proxy variables for investor sentiment [10]. Their empirical results found that when the stock price rises, investors will be influenced by skeptical sentiments and short the

362

M.-H. Liao et al.

lending securities. But when the stock price continues to rise, short sellers will lose money. This means that investors are susceptible to sentiment, which may make them less profitable in the market. The study sample is daily data during 2007. Ou used the market turnover rate, the open interest of the Taiwan index, the turnover rate of the institutional investors, institutional investors’ net buy/sell, futures trading volume of Taiwan index by the institutional investors, securities lending /borrowing rate, trading volume of odd lot and net buy/sell of Taiwan index futures traded by institutional investors and individual investors are used as proxy variables for investor sentiment [11]. And the results show that when the investors have an optimistic view on the market, fund’s returns will decline. Monthly data were used for this study sample. The study period was from 2006 to 2017. Lin et al. used China’s consumer confidence index as an indicator to measure investor sentiment and found that when investor sentiment was high, analysts would give more favorable ratings, and analysts who predicted market conditions in the market will increase [12]. The study sample is annual data from 2004 to 2009. Yeh and Li used the total turnover rate series as a proxy variable of investor sentiment and found that when the market was in a bull market, investor sentiment was positively correlated with returns [13]. But when the market bear, investor sentiment and returns are negatively correlated. Lin et al. also used the turnover rate series as sentiment indicator [14]. Their findings show that there is sentiment momentum in the market. And the sentiment momentum is related to the rate of return. Their sample is monthly data from 1982 to 2007. Hsu et al. divided sentiment indicators into two categories: trading activities and derivative financial markets [15]. And they found that Stock price volatility has an impact on financing behavior. Changes in financing amount are related to fluctuations in previous returns. Compared with institutional investors, the sentiment of individual investors in the Taiwan stock market is more easily affected by market fluctuations. Their sample for this study is daily data and intra-day minute-by-minute data from January 5, 2000 to December 5, 2003. Some studies have classified investor sentiment as an irrational factor in noise trading [16, 17]. 2.2 Growth and Value Stocks Many studies in academia define value stocks and growth stocks by using the ratio of accounting data to the company’s market price. Erin H. C. Kao et al. evaluated value investment strategies based on three ratios: price-earnings ratio, market-to-book ratio and market-to-sales ratio [18]. They found that buying at relatively low levels and selling at relatively high levels based on the marketto-sales ratio and market-to-book ratio of each stock significantly improved annualized returns. The study period was from January 2000 to December 2011. Uang and Yu used the price-earnings ratio and market-to-book ratio as an indicator to measure business growth [19]. And they found that financial analysts can more correctly predict the price-earnings ratio of growing companies. The study sample is quarterly data from 1997 to 2001. Penman used the price-earnings ratio and market-tobook ratio to define growth companies, and found that the price-earnings ratio reflects future profitability and is an effective predictor of profit growth [20].

Comparing Investor Sentiment Between Growth and Value Stocks

363

2.3 Portfolio and Investor Sentiment Tsai et al. found that under the difference between high and low investor sentiment, the company’s financial and accounting characteristics have different effects on stock returns [21]. During the period of low investor sentiment, earnings are invested in portfolios with high book-to-equity ratios. Returns are higher than for portfolios with a low earningsto-book equity ratio. And the current investor sentiment is positively correlated with the current return. The previous investor sentiment is negatively correlated with the current return. We can find from the above literature that when the company characteristics are different, the response of investor sentiment is also different. Similar differences in investor sentiment response should also exist between growth stocks and value stocks. So the following hypothesis is established: H1: Investor sentiment in growth stocks is high relative to value stocks as the market price rises. H2: Investor sentiment in value stocks is stable relative to growth stocks as the market price falls. H3.1: Investor sentiment in previous issue growth stocks has a positive impact on current excess returns. H3.2: Investor sentiment in previous issue value stocks has a positive impact on current excess returns.

3 Research Methods This research mainly discusses the relationship between investor sentiment and stock returns. Therefore, the samples selected in this research are listed companies in Taiwan stock market as the research object. The research period is from January 2008 to December 2021. In this study, stock prices, and trading volumes as institutional investors’ sentiments are obtained from the Taiwan Economic Journal (TEJ) database. The authoritative data bank covering extensive institutional investors’ trading data sets in Taiwan Stock Exchange (TWSE) since 2008. We explore differences in investor sentiment for different company characteristics. The proxy variables of investor sentiment include the turnover rate of 3 major institutional investors (TORE), securities lending/borrowing rate (LBRATIO) and net buy/sell of futures by 3 major institutional investors (TIFL). And we measure company growth by price-earnings ratio and market-to-book ratio. We classify stocks of a company whose price-earnings ratio or market-to-book ratio is above market level as growth stock. And the stocks of a company whose price-earnings ratio or market-to-book ratio is below market level as value stock. Investor sentiment differs between growth and value stocks when the market is up. Therefore, we set up two models to observe the changes separately. Model 1: Sentimentt=up,growth stocks = Sentimentt=up,value stocks

364

M.-H. Liao et al.

Model 2: Sentimentt=down,growth stocks = Sentimentt=down,value stocks To observe whether the investor sentiment of growth (or value) stock in the previous period affects the current return, we use the excess return (ER) that exceeds the market return rate as a proxy variable for the return. GROWTHi is the dummy variable with value of 1 for the ith company whose price-earnings ratio or market-to-book ratio is above market level; and 0 otherwise, Model 3: ERit = a + Sentimentijt−1 ∗ GROWTHijt−1 + eit So far there is little investor sentiment research on value stocks and growth stocks. It is believed that this research will contribute to a more complete understanding of investor behavior patterns in academia. Acknowledgments. Constructive comments of editors and anonymous referees are gratefully acknowledged. This research is partly supported by the National Science Council of Taiwan (NSC 109-2813-C-468-099-H and NSC 110-2813-C-468-086-H).

References 1. Kahneman, D., Tversky, A.: Prospect theory: an analysis of decision under risk. Econometrica 47, 263–292 (1979) 2. De Bondt, W., Tversky, A.: Does the stock market overreact? J. Financ. 40, 793–805 (1985) 3. Shefrin, H., Statman, M.: the disposition to sell winners too early and ride losers too long: theory and evidence. J. Financ. 40, 777–790 (1985) 4. Weber, M., Camerer, C.F.: The disposition effect in securities trading: an experimental analysis. J. Econ. Behav. Organ. 33, 167–184 (1998) 5. Odean, T.: Are investors reluctant to realize their losses? J. Financ. 53, 1775–1798 (1998) 6. Froot, K.A., Scharfstein, D.S., Stein, J.C.: Herd on the street: informational inefficiencies in a market with short-term speculation. J. Financ. 47, 1461–1484 (1992) 7. De Long, J.B., Shleifer, A., Summers, L.H., Waldmann, R.J.: Noise trader risk in financial markets. J. Polit. Econ. 98, 703–738 (1990) 8. Brown, G.W., Cliff, M.T.: Investor sentiment and the near-term stock market. J. Empir. Financ. 11, 1–27 (2004) 9. Hirshleifer, D., Shumway, T.: Good day sunshine: stock returns and the weather. J. Financ. 58, 1009–1032 (2003) 10. Huang, P.Y., Ni, Y.S., Lai, P.S.: The interaction relationship between stock market information disclosed and the reaction of investors’ sentiments. Taiwan Financ. Q. 12, 115–144 (2011) 11. Ou, Y.T.: The study on the relationship between investor sentiment and fund performance. Contemp. Bus. Manag. 3, 74–96 (2018) 12. Lin, M.F., Chin, C.L., Chang, S.H.: Investor sentiment and analyst behavior. J. Manag. 28, 447–474 (2011) 13. Yeh, C.C., Li, C.A.: The Interaction between Investor Sentiment and Stock Returns. The development of the securities market Quarterly, Special Issue on Behavioral Finance, 153–190 (2019)

Comparing Investor Sentiment Between Growth and Value Stocks

365

14. Lin, C.P., Li, C.A., Yeh, C.C.: The relationship between investor sentiment and price momentum. Manag. Syst. 19, 729–759 (2012) 15. Hsu, H.N., Kuo, W.H., Cheng, N.C.: The interaction between investor sentiment and stock return fluctuation: an empirical study of Taiwan stock market. Taiwan Financ. Q. 6(3), P107– 121 (2005) 16. Peress, J., Schmidt, S.: What matters to individual investors? Evidence from the Horse’s Mouth. J. Financ. 75, 1083–1133 (2020) 17. Choi, J.J., Robertson, A.: Glued to the TV: distracted noise traders and stock market liquidity. J. Financ. 75, 1965–2020 (2020) 18. Erin, H., Kao, C., Wang, Z.W., Chen, C.W.: The study of value investing strategy-the application of P/E, P/B and P/S ratios. Lingdong J. 1–35 (2013) 19. Uang, J.Y., Yu, C.H.: Financial analysts’ forecast errors and behavioral factors affecting earnings forecasts. J. Securities Market Dev. 16, 117–144 (2005) 20. Penman, H.S.: The articulation of price-earnings ratios and market-to-book ratios and the evaluation of growth. J. Account. Res. 34, 235–259 (1996) 21. Tsai, P.J., Wang, Y.J., Chang, C.C.: Investor sentiment firm characteristics, and stock returns in Taiwan. Econ. Res. 45, 273–322 (2009)

Application of TOWS Matrix Analysis in a Precision Medicine Genetic Testing Company Kuei-Yuan Wang1 , Ying-Li Lin1(B) , Chien-Kuo Han2 , and Chia-Wei Eddie Liang3 1 Department of Finance, Asia University, Taichung, Taiwan

{yuarn,yllin}@asia.edu.tw

2 Department of Food Nutrition and Healthy Biotechnology, Asia University, Taichung, Taiwan

[email protected]

3 Gene Target Group, Kaohsiung, Taiwan

[email protected]

Abstract. This study uses a Taiwanese precision medical genetic testing company as a sample. The company’s market share ranks among the top three in Taiwan. This study conducted in-depth interviews with the company’s CEO using a case study method, and analyzed it through SWOT and TOWS matrix analysis. It is hoped that through the results of this study, some suggestions on business strategies for Taiwan precision medical genetic testing companies will be provided.

1 Introduction The world has entered a new era of AI artificial intelligence. Big data analysis is gradually infiltrating all walks of life, bringing technological changes and increasing efficiency, and improving scientific accuracy. The medical industry has also entered the era of AI, which is accelerating the new century of precision medicine. U.S. bets up to $215 million to pursue the Precision Medicine Initiative project. The UK also launched the Precision Medicine Catapult project. China also sees precision medicine as a priority industry. It is obvious that countries have increasingly focused and invested in the precision medicine industry. Precision medicine is a method of treating and preventing disease. Precision medicine is a personalized approach to disease rather than a one-size-fits-all approach. It considers factors that may influence health disparities, such as age, ethnicity, place of residence, habits and family health history. Through precision medicine, patients can be informed about the most appropriate way to maintain their health and find the most suitable treatment (Griffin and Eric 2018). According to the National Institutes of Health (NIH), precision medicine is an emerging approach to disease treatment and prevention. Precision medicine examines not only individual genes, but also living environment and lifestyle to assess individual differences. Examining individual genes and related assessments will allow physicians and researchers to make more precise predictions of treatment and prevention strategies for specific diseases, and which population groups they may apply. It is very different from the traditional way of treating various people with one medicine. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 366–375, 2022. https://doi.org/10.1007/978-3-031-08819-3_39

Application of TOWS Matrix Analysis

367

Precision medicine makes extensive use of big data analysis of genetic variation to assist R&D institutions and medical institutions to improve disease prevention and cure rates based on subtle differences at the molecular level (NIH 2018). This study tries to achieve the following purposes. (1) Use SWOT analysis to analyze the internal and external environment of the sample company. (2) Use TOWS matrix analysis to put forward operation suggestions for the sample company.

2 Methodology 2.1 Case Study Yin (1981) indicated that case study method is one of the basic research methodologies in the social sciences. He also indicated that case study should focus on the analysis of the sample company, rather than merely gathering the information of the sample company. Ye (2001) also indicated that case study method is a way to gather complete information for a particular individual or group, and then conduct an in-depth analysis of the cause-effect relationship of their problems. 2.2 Interview Outline The interview outline of this study was designed by the unstructured interview method. This study utilized many methods to prevent the confusion in semantic terms of the interviewee. For example, avoiding using too difficult academic terms, choosing colloquial words, and giving examples to help the interviewee to understand the problem more easily while interviewing. The interview outlines of this study were as follows: a. Who are the main competitors of your company? b. What are your company’s strengths and weaknesses compared to your competitors? c. What are the opportunities and threats faced by the precision medical genetic testing industry? d. Facing the competitive industrial environment, what strategies does your company have to deal with? 2.3 Interviewee This study interviewed the CEO of a precision medicine genetics company in Taichung on November 17, 2021.

3 Analysis Results 3.1 Introduction of the Sample Company The case company was established in 2012. It focuses on the fields of gene synthesis, expression proteins, antibody customization services, gene sequencing analysis and other fields. The number of employees is about 30 to 50 people. The capital amounted to NT$30

368

K.-Y. Wang et al.

million. Business locations include Taipei, Taichung and Kaohsiung in Taiwan. The case company also established a laboratory in a national university incubator center. The company’s vision is to become a technology leader in the precision medicine genetic testing industry and to promote the popularization of precision medicine. Their mission is to effectively prevent diseases, accurately provide cancer treatment drugs, create a big data platform for medical care and health, and enhance the new era of digital health. 3.2 SWOT Analysis Results The results of the SWOT analysis can provide the choice of business strategy and assist the business in establishing and utilizing the company’s strengths, reducing or reducing weaknesses, taking advantage of external opportunities, and reducing external threats. The SWOT analysis results of the case companies are shown in Table 1. Table 1. SWOT analysis results of the sample company Strength

Weakness

a. Next-generation gene sequencing analysis and precision medicine b. Establish a digital medical platform c. Build a large database d. eHealthcare APP e. Core technology of antibody development and production f. Development and assembly of various rapid screening reagents g. Patented biomarker analysis platform h. Global biomedical innovation ecosystem i. Academic research combined with clinical research j. First entrant advantage k. Lifetime updates of the FDA Personal Medication Database l. Continuous talent development

a. Equipment suppliers monopolize the market b. Established for a short period of time c. Big data input costs are high and recovery is slow d. Cross-unit data database integration is not easy e. Lack of domestic talent

(continued)

Application of TOWS Matrix Analysis

369

Table 1. (continued) Strength

Weakness

Opportunity

Threat

a.Barriers to new market entrants b.Precision medicine at home and abroad has entered a growth period c.Oncology Market by Gene Sequencing d.There is huge room for growth in the Chinese market e.Amendment of specific medical technical inspection and instrument use management methods f.Small and medium hospitals can be the target market g.The preventive medicine market is booming h.U.S. approves breast cancer mutation gene detection kits for direct sale i.The issue of precision medicine is included in the regulations on the development of new biotechnology and new drug industries

a. Threats from domestic and Asian competitors b. Information security issues such as privacy rights and information outflow affect the sources of medical big data c. Hospitals set up genetic testing services d. Highly competitive industry and the price elasticity of each service is high

3.2.1 Strength a. Next-generation gene sequencing analysis and precision medicine i. Preventive Medicine: Provides one-time testing for a lifetime of services. ii. Cancer precision medicine: There are many genetic testing items, which can detect more than tens of thousands of gene mutation sites. And based on the detection of individual differences in patients’ response to drugs, precise medication is provided. iii. Post-operative tracking: accurate tracking and detection to correctly provide postoperative treatment medication. b. Establish a digital medical platform Next-generation gene sequencing services require more accurate sequencing information. Under normal circumstances, detecting a person’s genome generates about 90Gb of data, which cannot be carried by general servers. In addition, excellent computing power is required to analyze and compare more than 400,000 samples. The establishment of a database is of positive significance for the interpretation of various clinical diseases. Therefore, the case company has established a cloud data platform to accurately analyze and interpret personal genomic data. c. Build a large database Sample companies have built their own large databases and developed relevant software for rapid data analysis. It enables integrated analysis of big data to interpret

370

d.

e.

f.

g.

h.

i.

j.

k.

l.

K.-Y. Wang et al.

genetic testing results. The database of the case company includes Asian, European and American databases, covering hospitals, medical centers and FDA databases. eHealthcare APP In order to make it easy for the general public to understand, the case company developed the eHealthcare APP. By reading graphic electronic reports and enjoying interactive online explanations, the public can keep health information at their fingertips at any time. The eHealthcare APP also provides online medication safety consultation to create a community pharmacy concept and allow pharmacists to interact online in real time. Core technology of antibody development and production The company’s high-quality antibody research and development technology can not only provide academic research and clinical testing, but also provide pet disease and food safety monitoring. Development and assembly of various rapid screening reagents In addition to genetic testing, the R&D team also develops rapid screening reagents, as well as food and drug testing reagents. Patented biomarker analysis platform With the single-molecule drug analysis platform patented in Taiwan and the United States, it can accelerate the development of new drugs in pharmaceutical companies and applications such as immunotherapy, as well as perform effectiveness analysis. Global biomedical innovation ecosystem In March 2018, the case company joined the Digital Health International Accelerator to establish links with the international industry. The case company also won the Digital Health International Accelerator Silver Award at the Digital Health Forum of the GEC+ Global Entrepreneurship Conference in September 2018. Academic research combined with clinical research The case company has a foundation of more than 7 years in academic research. The case company also cooperates with domestic and foreign universities and research institutes to provide a full range of customized commissioned services. The case company is good at antibody development and various biomarker screening. Case companies can assist academic research institutes or enterprises in the development of leading rapid screening reagents. Through the combination of clinical and academic research resources, a major advantage of the case company is formed. First entrant advantage Limited by the limited number of hospitals, the first entrants will definitely have more advantages. Since the case company is an early market entrant, it has a greater first-entrance advantage. Lifetime updates of the FDA Personal Medication Database Next-generation gene sequencing coupled with big data analysis facilitates the feasibility of precision medicine. The case company obtained a new authorization in the FDA’s personal medication database, allowing customers to use one test for a lifetime. Continuous talent development

Application of TOWS Matrix Analysis

371

The case company not only employs outstanding graduates and doctors in the fields of medicine, biochemistry and bioinformatics, but also provides a comprehensive welfare system, so the turnover rate is low. Through the combination of professional technology and business, the case company can better win the trust of customers in the profession. 3.2.2 Weakness a. Equipment suppliers monopolize the market The high cost of gene sequencing detection instruments and related equipment has led to the monopoly of European and American companies in the instrument and consumables market. Markets other than Europe and the United States must accept high-cost imports of core equipment, consumables and reagents, forming a disadvantage for individual companies. b. Established for a short period of time The case company was established in 2012 and has only been established for about 10 years. Therefore, for the precision medical genetic testing industry that needs to accumulate experience and professionalism, case companies still have a lot to learn and grow. c. Big data input costs are high and recovery is slow The combination of medical treatment and the Internet has become the development trend of the precision medicine industry. Genetic testing requires a large amount of data, and analyzing the results of the test requires a larger comparison database. The establishment of massive data collection and comparison databases requires high up-front costs, and the return on investment is relatively slow. d. Cross-unit data database integration is not easy How to integrate cross-unit data databases will determine the development of precision medicine. If a meaningful disease treatment context can be analyzed and summarized, it can also be of great value. Because of the diversity of sources and methods of information collection, quality control of this data is a challenge. And when information from different institutions, regions or collection methods is pooled together, integrating complex information is also an important task. e. Lack of domestic talent Data sorting and archiving can be done by general data professionals, but there are not enough medical professionals who can really analyze the huge data. These data must be converted into information that physicians, patients and their families can understand, in order to truly reap the benefits of precision medicine and further implement personalized treatment. However, there is a shortage of talent in this area in the country. 3.2.3 Opportunity a. Barriers to new market entrants Precision medical genetic testing requires professional knowledge in the fields of medical testing and biotechnology, and the threshold for professional technology and capital is relatively high. Even if new entrants are well-funded, the amount of

372

b.

c.

d.

e.

f.

g.

h.

K.-Y. Wang et al.

genetic data of patients is far less than the amount of data accumulated by existing entrants. Therefore, entry barriers for new entrants are numerous and large. Precision medicine at home and abroad has entered a growth period Countries have invested significant resources towards precision medicine. Taiwan’s precision medicine has been recognized by the world as one of the indispensable and important countries in the new generation of preventive medicine. The development of precision medicine at home and abroad and the support of government resource units will help to develop more new customers or potential customers. Oncology Market by Gene Sequencing Tumor diagnosis and treatment is an application market with great growth momentum of gene sequencing. Changes in life and habits in recent years have resulted in an increase in the global cancer incidence rate year by year. The burden of cancer medical expenses is increasing the government’s support for the genetic industry. Therefore, it is another positive factor for the development of the precision medical genetic testing industry. There is huge room for growth in the Chinese market China’s population accounts for about one-fifth of the world’s population, and there are sufficient and abundant samples and huge potential market demand for common and rare diseases. Under the influence of factors such as population aging, environmental pollution and urbanization, the number of cancer cases in China has gradually increased. The scale of China’s precision medicine market is growing rapidly. Amendment of specific medical technical inspection and instrument use management methods Through the revision of relevant measures, in the future, medical institutions can formulate specific implementation plans in accordance with relevant measures, and after being approved and registered by the Ministry of Health and Welfare, patients who meet the indications can be given cell therapy. Small and medium hospitals can be the target market Due to limited funds and scale, small and medium-sized hospitals rarely set up gene sequencing services on their own. Therefore, small and medium-sized hospitals need to cooperate with external units. For the precision medical genetic testing industry, new target market opportunities will be formed. The preventive medicine market is booming High-quality and high-standard health examinations are gradually accepted by the market, and the number of people willing to accept high-level health examinations has greatly increased. Large-scale hospitals and professional health check-up clinics in Taiwan have invested in the high-end health check-up market. They mostly attract self-paying customers with high efficiency and high level of service. U.S. approves breast cancer mutation gene detection kits for direct sale In March 2018, the U.S. Food and Drug Administration (FDA) approved the genetic testing company 23andMe to sell genetic testing reagents directly to consumers to satisfy the desire of certain people to more clearly understand their own health information. . Compared with traditional standardized treatment, precision medicine is closer to the concept of personalized treatment, and the needs of patients are also considered during treatment.

Application of TOWS Matrix Analysis

373

i. The issue of precision medicine is included in the regulations on the development of new biotechnology and new drug industries With the promotion and support of national policies, there are more development opportunities for the precision medicine industry. In order to implement the five major industrial innovation plans and build Taiwan into the Asia-Pacific biotechnology and pharmaceutical R&D industry center, the draft amendments to some provisions of the Biotechnology and New Drug Industry Development Regulations were announced and implemented in January 2017. The publication and implementation of this regulation will contribute to the development of the precision medicine industry. 3.2.4 Threat a. Threats from domestic and Asian competitors There are not only many major competitors in Taiwan, but also the scale and services are roughly the same with the sample company. The strength of each company is comparable, and there is a high degree of competition in the market. b. Information security issues such as privacy rights and information outflow affect the sources of medical big data Precision medicine involves a lot of personal private information, causing customers to worry that these medical data will be used improperly and affect their willingness to participate. c. Hospitals set up genetic testing services The current business models of genetic testing services can be divided into two categories: hospital models and third-party testing models. In December 2018, a large domestic cancer center hospital was officially completed and provided genetic testing services. Customers may consider the convenience of follow-up treatment and choose the testing service established by the hospital from the beginning. This leads to a competitive threat to third-party genetic testing companies. d. Highly competitive industry and the price elasticity of each service is high There are more than 200 gene sequencing service companies in Taiwan. In order to increase the number of customers, customer discount programs are often provided. Customers can compare prices through online information and decide to choose a more favorable company. As a result, the precision medical genetic testing industry has high competition pressure and high service price elasticity. 3.3 TOWS Matrix Analysis Results Based on the above SWOT analysis results, this study tried to utilized the TOWS matrix analysis to form some suggestions to the sample company. The TOWS matrix analysis results were showed in Table 2.

374

K.-Y. Wang et al. Table 2. TOWS analysis results of the sample company Opportunity

Threat

Strength

SO strategy: a. Planning to enter the mainland and Southeast Asian markets b. Develop small and medium-sized hospitals as target customers c. Cooperate with health examination center to promote gene sequencing test d. Adopt joint venture or IPO model to increase funding sources e. Comprehensive testing services for the tumour market

ST strategy: a. Improve information security protection and risk management standards b. Continue to add eHealthcare APP functions and optimize them c. Continuously develop new products d. Apply for more patents to boost your competitive advantage d. Cooperation with academic research units e. Collaborate with hospitals

Weakness

WO strategy: a. Introduce marketing talents to the precision medical genetic testing industry ba. Improve R&D technology innovation ability c. Nurturing professional consultants

WT strategy: a. Temporarily stop R&D, production, and sales of products that have made profits and losses for 3 consecutive years b. For products that have not yet made a profit, look for cooperative units to improve product functions or added value

4 Conclusions Precision medicine is gradually maturing, and the market is showing a growth trend. Based on the results of the above SWOT analysis and TOWS matrix analysis, the business strategy recommendations for the case companies are as follows: (1) Planning to enter the mainland and Southeast Asian markets; (2) Develop small and medium-sized hospitals as target customers; (3) Cooperate with academic research units, hospitals and health examination centers and other units; (4) Adopt joint venture or IPO model to increase funding sources; (5) Continuously develop new products and provide all-round high-quality testing services; (6) Continuously optimize eHealthcare APP functions; (7) Introduce marketing talents; (8) Improve information security protection and risk management standards; (9) Improve R&D technology innovation ability; (10) Cultivate professional consultants; (11) Temporarily stop R&D, production, and sales of products that have made profits and losses for 3 consecutive years; (12) For products that have not yet made a profit, look for cooperative units to improve product functions or added value. This study has the following research limitations and recommendations. This research is only based on case studies. The samples cannot cover all fields of precision medicine in Taiwan, and the number of related industries listed on the OTC market is relatively small, so it is impossible to obtain financial report information, and it is impossible to make suggestions on the future development trend of the industry. A holistic study of the industry. In addition, this study uses the interview method to explore, and it is suggested that the questionnaire survey of consumers can be extended in the future.

Application of TOWS Matrix Analysis

375

By understanding the needs and suggestions from the consumer side, it can also be used as a business strategy and development suggestion for operators. Due to the lack of precision medicine research literature, and most of them are combined in biotechnology medical related research, there are few researches on precision medicine. Therefore, there are still differences in the evaluation, which may affect the accuracy of the research results. In addition, this study only focuses on the precision medicine industry. It is suggested that future researchers can conduct in-depth research on the main competing countries of the precision medicine industry in Asia, and explore the impact of international precision medicine on the promotion and development of precision medicine in Taiwan. This study is believed to contribute more to Taiwan’s future precision medicine strategy. Acknowledgments. Authors thank anonymous reviewers for their valuable suggestions.

References Griffin, R., Eric, D.: What is precision medicine? Healthy Moments Episode. Retrieved Aug 05, 2018, from https://www.niddk.nih.gov/health-information/healthy-moments/episodes/what-isprecision-medicine (2018) NIH.: What are some potential benefits of precision medicine and the precision medicine initiative? Retrieved Aug 21, 2018, from https://ghr.nlm.nih.gov/primer/precisionmedicine/potentialben efits (2018) Ye, C.C..: Educational Research Method. Taipei: Psychology (2001) Yin, R.K.: The case study crisis: some answers. Adm. Sci. Q. 26(1), 58–65 (1998)

Research on the Intention of the Elderly to Participate in Barrier-Free Tours Ya-Lan Chan1 , Wen-Qian Li1 , Sue-Ming Hsu2 , and Mei-Hua Liao3(B) 1 Department of Business Administration, Asia University, Taichung, Taiwan 2 Department of Business Administration, Tunghai University, Taichung, Taiwan 3 Department of Finance, Asia University, Taichung, Taiwan

[email protected]

Abstract. In recent years, Taiwan’s tourism market has become very popular, but in terms of market segmentation, the promotion of a more diverse tourism market still needs to be developed. At present, Taiwan has entered an aging society. If there are professional tourism services designed for “barrier-free” people, it will be able to promote the well-being of a successful aging society. For the tourism industry, this is a also potential business opportunity. This study used a questionnaire to investigate the people with disabilities and family caregivers, and conducted descriptive statistical analysis and regression analysis on the popularity and perceived risk of tourism services. Travel planning for both parents and their caregivers concluded that perceived risk had a limited negative impact on willingness to purchase accessible travel, while wellknown brand’s image and service quality could significantly improve purchase willingness.

1 Introduction The Economic Development Association of the Executive Yuan estimates that the 65year-old population in Taiwan will account for the total population in 2025 20.1%, that is, one elderly person in every five people, has reached the standard of a super-aged society. The Ministry of Health and Welfare predicts that in 2031, the disability rate of people over 65 years old in Taiwan will be as high as 16.3%.It means that in the future, there will be as many as 1.2 million disabled elderly people in Taiwan. Such a considerable number does not even include persons with disabilities caused by birth or major accidents. According to the statistics of the Taiwan Ministry of Health and Welfare, the number of disabled people in Taiwan reached 366,781 in 2016. However, the barrier-free shuttle services provided by the government in the past was aimed at solving basic needs, and rarely took care of the disabled people’s right to entertainment. Silver-haired elders and people with disabilities are unable to travel around Taiwan and experience the beauty of Formosa due to their limited mobility. Therefore, we want to study a travel agency specially designed for silver-haired elders and disabled people. Many people have never been in contact with this type of travel agency, so consumers will associate and infer the quality of products based on their impressions, attitudes and feelings about © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 376–381, 2022. https://doi.org/10.1007/978-3-031-08819-3_40

Research on the Intention of the Elderly to Participate

377

the brand. When consumers have doubts about a product, the brand will Image will be one of the important indicators for evaluating products and influence consumers’ purchasing decisions (Gardner and Levy 1955; Newman 1957; Jacoby et al. 1971). Zins (2001) research pointed out that good product quality will drive consumers’ good image of the company. Haoer and Shufang (2002) pointed out that service quality has a significant positive impact on brand image. If the company has a good brand image, it will help increase the willingness of consumers to buy (Walters 1974). The more unfamiliar consumers are with the product, the higher the perceived risk.

2 Related Research on the Purchase Intention of Barrier-Free Tours According to Park et al. (1986), the concept of brand image is refer to a conceptual management framework that was used to measure the content of the questionnaire, and this study adjusted its volume item. (1) Functional: Emphasizes products or services designed to help consumers solve existing problems, preventing potential problems that may occur, such products can solve consumers’ basic needs. (2) Symbolic: Emphasizes the product brand that can meet the internal needs of consumers. To enhance self-worth, role positioning, and self-identity, such products are used as personal connections to a specific group or to improve self-image. (3) Experience (experiential) emphasizes that it can satisfy consumers’ desires and provide perceptual fun such products are used to meet the needs of consumers seeking diversification. For the service quality, Parasuraman et al. (1988) argued that the service quality model consists of five dimensions: (1) Tangibles: the grooming and communication of service personnel, and the equipment. (2) Reliability: The ability of service personnel to reliably provide promises. (3) Responsiveness: the agility of service personnel to provide services, and willingness to help customers. (4) Assurance: The service personnel have sufficient service knowledge and are friendly twenty-three and gain the trust of customers. (5) Empathy: The service personnel provides sufficient care and attention to consumers’ meaning. For the perceived risk, it was originally extended from psychology by scholar Bauer (1960), who believed that consumers could not predict the results before purchasing a product, or faced the situation of whether the consumption is correct, and then affect the decision-making of shopping. Perceived risk is mainly divided into six major components: (1) Time Risk: The time consumers waste when consuming (Roselius 1971). (2) Financial Risk: When the product is defective or unusable, consumers will Monetary losses for repairs, and replacements (Roselius 1971). (3) Performance Risk: The loss of function caused by the use results that are not as expected until after consumption (Jacoby and Kaplan 1972b). (4) Social Risk: When the purchased product cannot satisfy others, or cannot Brings social value identification (Jacoby and Kaplan 1972a). (5) Physical Risk: If the product is of poor quality, it may cause health and safety risks (Roselius 1971). (6) Psychological Risk: Consumers buy products that do not meet the psychological or self-perceived damage at expected levels (Roselius 1971). For purchase intentions, Dodds et al. (1991a, 1991.b) based on the possibility of consumers buying the product, whether they would consider buying the product and whether

378

Y.-L. Chan et al.

it is recommended to others are used as variables to measure purchase intention. Sihi et al. (1998) and other scholars proposed that to measure consumers’ purchase intention, consumers’ loyalty should be measured, including the willingness to repurchase, the willingness to buy more products in the future, and the willingness to recommend to others willing. Schiffman and Kanuk (2000) use five measurement options: I buy, I might buy, I’m not sure, I probably don’t buy, and I don’t buy. Zhuang Limin (2006) pointed out that measuring the willingness to buy can be divided into three aspects: the possibility of buying a product, considering it, and recommending others to buy it.

3 Research Methodology Our study adopted a convenience-sampling questionnaire. We analyze the data by SPSS software, including descriptive statistics analysis and multiple regression analysis to obtain research results. The research structure is as Fig. 1.

Fig. 1. Research structure of the study

H1: Consumers’ service quality has a positive impact on the brand image. H2: Consumers’ service quality has a negative impact on perceived risk. H3: Consumers’ brand image has a negative impact on perceived risk. H4: Consumers’ perceived risk has a negative impact on purchase intention. H5: Consumers’ brand image has a positive impact on purchase intention.

4 Data Analysis After reliability, validity, and factor analysis, the dimension of service quality can be divided by Reliability and empathy, the dimension of brand image can be divided by functional and experiential, and the dimension of perceived risk can be divided by internal loss and external loss. After multiple regression analysis, the results of our regression analysis are shown in Table 1.

Research on the Intention of the Elderly to Participate

379

Table 1. Result of the hypothesis

5 Conclusions There are two ways to increase purchase intention: 1. Reduce perceived risk Launching promotional programs From the perceived risk of the scale, Customers are most concerned about spending more. The price of the family holiday is beyond the budget, so it is recommended to launch the "Family Traveler" program to enhance the disabled people’s willingness to travel. Design a physical load test scale Many consumers worry that the traveling program planned by Duo Fu Holiday will be too hurried and tiring for the elders. Therefore, we suggest that Duo Fu Holiday can cooperate with medical institutions and design several physical loads for the elders. The scale allows consumers to choose the most suitable program according to their physical fitness level to reduce their physical risks. 2. Improve brand image Adding service bases According to the population statistics from the Department of Household Affairs of the Ministry of the Interior, Taipei and Kaohsiung are the areas with the highest population of elderly people over 65 years old in Taiwan. However, the base of tour companies are mostly in Taipei, so it is recommended that the tour companies set up a new base in Kaohsiung to make it easier for consumers to obtain information, thereby improving its service quality and improving its brand image, so as to increase consumers’ willingness to buy.

380

Y.-L. Chan et al.

References Agustin, C., Singh, J.: Curvilinear effects of consumer loyalty determinants in relational exchanges. J. Mark. Res. 42(1), 96–108 (2005) Andrew, J.D., Clayton, F.W., Charles, P.D.: The relationship between counselor satisfaction and extrinsic job factors in state rehabilitation agencies. Rehabilition Counseling Bull. 45(4), 223– 232 (2002) Arjun, C., Morris, B.H.: The chain of effects from brand trust and brand affect to brand performance: the role of brand loyalty. J. Mark. 65, 81–93 (2001) Bateson & Hoffinan, G.: Essential of Service Marketing: Concepts, Strategy, and Cases. Harcourt Inc. (2002) Bauer, R.A.: Consumer behavior as risk taking. In: Hancock, R.S. 86 (ed.) Dynamic Marketing for a Changing World. American Marketing Association, Chicago, pp. 389–398 (1960) Blackwell, R.D., Miniard, P.W., Engel, J.F.: Consumer Behavior, 10th edn. South-Western College, California (2006) Chaudhuri, A., Holbrook, M.B.: The chain of effects from brand trust and brand affect to brand performance: the role of brand loyalty. J. Mark. 65(2), 81–93. 14 (2001) Chen, T.Y., Chang, P.L., Chang, H.S.: Price, brand cues, and banking customer value. Int. J. Bank Mark. 23(3), 273–291 (2005) Chernatony and McWilliam: .Branding terminology the real debate. Market. Intell. Plann. 7(7/8), 29–32. 87 (1989) Dodds, B.W., Monroe, K.B., Grewal, D.: Effects of price, bands and store information on buyers’ product evaluation. J. Mark. Res. 28(3), 307–319 (1991) Dodds, W.B., Monroe, K.B., Grewal, D.: The effects of pice, brand, and store information on buyer’s product evaluations. J. Mark. Res. 28(8), 307–319 (1991) Jacoby, J., Kaplan, L.: The components of perceived risk. In: Proceedings of 3rd Annual Conference, Association for Consumer Research, pp. 382–393 (1972) Jacoby, J., Kaplan, L.: The components of perceived risk. In: Venkatesan, M. (ed.) 3th Annual Conference of the Association for Consumer Research, pp. 382–393. Association for Consumer Research, Chicago, IL (1972) Kang, G., James, J.: Service quality dimensions : an examination of 89 Gronroos’s service quality model. Manag. Serv. Qual. 14(4), 277–295 (2004) Kapferer, J.N.: Strategic Brand Management: New Approaches to Creating and Evaluating Brand Equity. The Free Press, New York (1992) Keller, K.L.: Strategic Brand Management: Building, Measuring , and Managing Brand Equity. Pearson Prentice Hall, Upper Saddle River (2008) Keller, K.L.: Building customer-based based brand equity. Mark. Manage 10(2),14–19 (2001) Morgan, R.M., Hunt, S.D.: The commitment-trust theory of relationship marketing. J. Mark. 58(3), 20–38 (1994) Nguyen, N.O., LeBlance, G.: Corporate image and corporate teputation in customer’s retention decision in service. J. Retailing Consumer Serv. 8(4), 227–236 (2001) Parasuraman, A., Zeithaml, V.A., Berry, L.L.: A conceptual model of service quality and its implications for future research. J. Mark. 49, 41–50 (1985) Park, C.W., Jaworski, B.J., Macinnis, D.J.: Strategic brand concept image management. J. Mark. Manag. 50(4), 621–635 (1986) Park, J.W., Robertson, R., Wu, C.L.: The effect of airline service quality on passengers’ behavioural intentions: a Korean case study. J. Air Transp. Manag. 10, 435–439 (2004) Roselius, T.: Consumer rankings of risk reduction methods. J. Mark. 35(1), 56–61 (1971) Schiffman, L.G., Kanuk, L.L.: Consumer behavior, 7th. Prentice Hall, NY (2000)

Research on the Intention of the Elderly to Participate

381

Sihi, N., McLaughlin, E.W., Wittink, D.R.: A model of consumer perception and store loyalty intentions for a supermarket retailer. J. Retailing 74(2), 223–245 (1998) Sirdeshmukh, D., Singh, J., Sabol, B.: Consumer trust, value, and loyalty in relational exchanges. J. Mark. 66, 15–37 (2002) Syed Alwi, S.F., Da Silva, R.V.: Online and offline corporate brand images: Do they differ? Corporate Reputatio (2007) Zins, A.H.: Relative attitudes and commitment in customer loyalty models: some experience in the commercial airline industry. Int. J. Serv. Ind. Manag. 12(3), 269–294 (2001)

Author Index

A Ampririt, Phudit, 1 Azuma, Masaya, 230 B Bai, Cuixia, 196 Barolli, Admir, 184, 238 Barolli, Leonard, 1, 10, 18, 31, 41, 54, 184, 211, 221, 230, 238 Batzorig, Munkhdelgerekh, 85 Bylykbashi, Kevin, 1, 184 C Chan, Ya-Lan, 360, 376 Chang, Chia-Chi, 275 Chang, Kuan-Chun, 275 Chen, Hsing-Chung, 247, 290 Chen, Lung-Pin, 297 Chen, Yen-Ju, 360 Chen, Ying-Ru, 297 Chiu, Shih-Ting, 307 Chung, Meng-Shao, 315 D Dao, Duyen Thuy, 120 Deng, Xinyang, 65 Duolikun, Dilawaer, 18 E Enokido, Tomoya, 18 G Gao, Tianhan, 65, 75, 161, 172 Guo, Nan, 65, 161, 196

H Han, Chien-Kuo, 366 Hashimoto, Hikari, 108 Hirata, Aoto, 41 Ho, Yu-Lun, 247 Hsiao-Yi, Ho, 334 Hsu, Sue-Ming, 376 Hsu, Wei-Chun, 290 I Ikeda, Makoto, 1, 10, 230 Ishida, Tomoyuki, 132 Islam, Md Rezanur, 85 J Jian, Yi-Cheng, 315 José, Agostinho António, 151 K Katayama, Kengo, 31 Kim, Jeena, 151 Kim, Jiha, 151 Kim, Myoungsu, 85 Kim, Seoyeon, 97 Kim, Yongho, 151 Kim, Yoonji, 97 Koh, Yeji, 97 Kulla, Elis, 211, 238 Kung, Tzu-Liang, 259, 267, 283 L Le, Anh Duy, 140 Leu, Fang-Yie, 307, 315, 325 Li, Wen-Qian, 376

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 L. Barolli (Ed.): IMIS 2022, LNNS 496, pp. 383–384, 2022. https://doi.org/10.1007/978-3-031-08819-3

384 Liang, Chia-Wei Eddie, 366 Liao, Mei-Hua, 360, 376 Liao, Mei Hua, 351 Lin, Cheng-Kuan, 283 Lin, Ying-Li, 344, 366 Liu, Yi, 238 Lu, Chai-Ju, 334 Lu, Yu-Syuan, 325 Luong, Huong Hoang, 120 M Matsuo, Keita, 1, 211 Mi, Qingwei, 172 Miwata, Masahiro, 10 N Nagai, Yuki, 31, 54, 221 Ngoc, Tai Le, 140 Nguyen, Hai Thanh, 120, 140 Nguyen, Khoa Thanh, 120 Nitama, Ryosuke, 132 O Oda, Tetsuya, 31, 41, 54, 221 Ogiela, Lidia, 207 Ogiela, Urszula, 207 Oh, Insu, 85, 97 P Park, Hyunhee, 151 Pham, Linh Thuy Thi, 120 Q Qafzezi, Ermioni, 1, 184 Qi, Jiayu, 75 S Saito, Nobuki, 41, 54 Sakamoto, Shinji, 184, 238 Sako, Seiya, 230

Author Index Shigeyasu, Tetsuya, 108 Sone, Hidekazu, 351 Susanto, Heru, 307, 315, 325 T Takizawa, Makoto, 18, 184, 207, 238 Teng, Yuan-Hsiang, 259, 267, 283 Thanh, Tan Nguyen Lam, 140 Torng, Chiou-Shya, 334 Toyoshima, Kyohei, 31, 41, 54, 221 Tran, Dien Thanh, 140 Tseng, Shian-Shyong, 247 Tsuneyoshi, Mitsuki, 10 U Uchimura, Shota, 230 Ueda, Chiaki, 221 V Van, Nhan Trong Pham, 120 W Wang, Kuei-Yuan, 344, 366 Wei, Ching-Chuan, 275 X Xie, Kang, 65 Y Yang, Ching-Ru, 344 Yang, Yunxia, 196 Yasunaga, Tomoya, 31, 54, 221 Yei, Leu-Fang, 297 Yim, Kangbin, 85, 97 Yu, Chiung-Wen, 360 Yukawa, Chihiro, 31, 41, 54, 221 Z Zhao, Cong, 161