Volume 8, Number 2, April 2022 IEEE Systems, Man and Cybernetics Magazine 1119693063
116 101 10MB
English Pages [51] Year 2022
Recommend Papers
File loading please wait...
Citation preview
IEEE SMC 2022 International Conference on Systems, Man, and Cybernetics Prague, Czech Republic October 9-12, 2022
“Integrating real world, virtual models, and society”
Call for Papers The 2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC 2022) is the flagship conference of the IEEE Systems, Man, and Cybernetics Society. It provides an international forum for researchers, educators, and practitioners to learn, share knowledge, report most recent innovations and developments, and exchange ideas and advances in all aspects of systems science and engineering, human-machine systems, and cybernetics. PAPER TEMPLATES (Word and LaTeX) SUBMISSION PAGE
Submission Guidelines Authors must adhere to the IEEE conference written paper format and they must submit the draft version of the paper for review before the due submission deadlines (see dates below). These are submitted through the PaperCept submission system (see link above). IEEE is very strict about the requirements for PDF files for inclusion in the IEEE Xplore® Digital Library. We strongly recommend using the set of templates in MS Word and LaTeX format provided by IEEE (see link above). Letter page size is preferred, and A4 can also be used. Please use them to create your paper, but don’t modify the style or the format under any circumstances. Submit only original work, not previously published or copyrighted. Prospective authors are expected to submit only their original works. The conference will be using the CrossCheck automated screening system to help verify the originality of papers. Submitted works may be compared to over 20 million articles in databases worldwide. Papers that violate IEEE’s publication principles may be summarily rejected. If the violation is deemed severe, then disciplinary actions may also be taken by IEEE. For further details please have a look into the Conference Authors @ IEEE Author Center. Accepted and presented papers will be copyrighted to IEEE and published in conference proceedings, which will be eligible for inclusion in the IEEE Xplore® Digital Library, once it meets the requirements of an IEEE quality review. It will then be customarily indexed by EI Compendex.
Regular Papers Prospective authors are invited to submit full-length papers electronically through the conference website. Papers (6 pages) should be concise but contain sufficient details and references to allow critical review. Papers will be reviewed by at least two referees for technical merit and content.
Special Sessions Papers Special Sessions provide a focused discussion of new and innovative topics. Special Session proposers should download the special session proposal template from the SMC 2022 website and submit the completed proposal to the Special Sessions Chairs. Special Session organizers should collect at least eight papers. All submitted papers undergo the same review process, and submission to proposed sessions is not a guarantee of acceptance.
Industrial Papers These contributions are intended to promote contributions from industries on technology development, innovations, and implementations, which will facilitate collaboration between industrial and academic members of the SMC community.
Digital Object Identifier 10.1109/MSMC.2022.3149439
www.ieeesmc2022.org
Workshop and Tutorial Papers These contributions are intended to promote contributions from applied research and applications, including work in progress, and facilitate increased collaboration between industrial and academic members of the SMC community.
BMI Workshop Papers The IEEE SMC 12th Workshop on Brain-Machine Interface (BMI) Systems will be held from October 9-12 as part of IEEE SMC 2022, the flagship annual conference of the IEEE Systems, Man, and Cybernetics Society. The goal of the Workshop is to provide a forum for scientists to present research results, facilitate the interaction and intellectual exchange between researchers, developers and consumers of BMI technology. We invite contributions reporting the latest advances, innovations and applications in BMIs. The BMI Workshop is organized by the IEEE SMC Technical Committee on Brain-Machine Interface Systems. Participation is free to all registered SMC 2022 attendees.
Deadlines Submission of Proposals for Special Sessions: February 15, 2022 Acceptance Notification of Special Sessions Proposals: March 15, 2022 Submission of Regular, Special Session, Industrial, BMI Workshop Papers, and Workshop and Tutorial Proposals: April 30, 2022 Acceptance Notification of Workshop and Tutorial Proposals: May 15, 2022 Acceptance Notification of Regular, Special Session, Industrial and BMI Workshop Papers: June 15, 2022 Final Camera-ready Submission of Regular, Special Session, Industrial, BMI Workshop, and Workshop and Tutorial Papers: July 15, 2022 Deadline for Early Registration: July 29, 2022 (single registration covers two submissions) Deadline for Late Registration: October 4, 2022
Topics Cybernetics (CYB)
Human-Machine Systems (HMS)
Systems Science & Engineering (SSE)
› Agent-Based Modeling
› Assistive Technology
› Communications
› Application of Artificial Intelligence
› Augmented Cognition
› Conflict Resolution
› Artificial Immune Systems
› Consumer and Industrial Applications
› Artificial Life
› Brain-based Information Communications
› Biometric Systems and Bioinformatics
› Design Methods
› Cooperative Systems and Control
› Cloud, IoT, and Robotics Integration
› Entertainment Engineering
› Decision Support Systems
› Complex Network
› Human Factors
› Discrete Event Systems
› Computational Intelligence
› Human Performance Modeling
› Distributed Intelligent Systems
› Computational Life Science
› Human-centered Learning
› Cybernetics for Informatics
› Human-Computer Interaction
› Electric Vehicles and Electric Vehicle Supply Equipment
› Deep Learning
› Human-Machine Cooperation and Systems
› Enterprise Information Systems
› Evolutionary Computation › Expert and Knowledge-Based Systems
› Human-Machine Interface
› Homeland Security
› Fuzzy Systems and their applications
› Information Systems for Design
› Infrastructure Systems and Services
› Heuristic Algorithms
› Information Visualization
› Intelligent Green Production Systems
› Hybrid Models of Neural Networks, Fuzzy Systems, and Evolutionary Computing
› Intelligence Interaction
› Intelligent Power Grid
› Interactive and Digital Media
› Intelligent Transportation Systems
› Interactive Design Science and Engineering
› Large-Scale System of Systems
› Image Processing and Pattern Recognition
› Control of Uncertain Systems
› Fault Monitoring and Diagnosis
› Kansei (sense/emotion) Engineering
› Information Assurance and Intelligence
› Manufacturing Automation and Systems
› Medical Informatics
› Intelligent Internet Systems
› Mechatronics
› Multimedia Systems
› Knowledge Acquisition
› Micro and Nano Systems
› Multi-User Interaction
› Machine Learning
› Modeling of Autonomous Systems
› Resilience Engineering
› Machine Vision
› Quality and Reliability Engineering
› Supervisory Control
› Media Computing
› Robotic Systems
› Systems Safety and Security
› Medical Informatics
› Service Systems and Organizations
› Multimedia Computation
› Team Performance and Training Systems
› Smart Buildings, Smart Cities and Infrastructures
› Neural Networks and their applications
› User Interface Design
› Smart Metering
› Optimization and Self-Organization approaches
› Virtual and Augmented Reality Systems
› Smart Sensor Networks
› Wearable Computing
› Soft Robotics
› Quantum Cybernetics
› System Architecture
› Quantum Machine Learning
› System Modeling and Control
› Representation Learning
› Technology Assessment
› Swarm Intelligence
› Trust in Autonomous Systems
› Transfer Learning
www.ieeesmc2022.org
Smart Solutions for Technology
www.ieeesmc.org
Volume 8, Number 2 • April 2022
Features 8 Frontiers of Brain-Inspired Autonomous Systems 21
How Does Defense R&D Drive the Innovations? By Ming Hou, Yingxu Wang, Ljiljana Trajkovic, Konstantinos N. Plataniotis, Sam Kwong, MengChu Zhou, Edward Tunstel, Imre J. Rudas, Janusz Kacprzyk, and Henry Leung
21 Home Energy Management Systems
Operation and Resilience of Heuristics Against Cyberattacks By Hafiz Majid Hussain, Arun Narayanan, Subham Sahoo, Yongheng Yang, Pedro H.J. Nardelli, and Frede Blaabjerg
31 Tensor-Based Knowledge Fusion and Reasoning for Cyberphysical-Social Systems
Theory and Framework By Jing Yang, Laurence T. Yang, Yuan Gao, Huazhong Liu, Hao Wang, and Xia Xie
39 Bitcoin Price Prediction in a Distributed
Environment Using a Tensor Processing Unit 39
A Comparison With a CPU-Based Model By Mohd Hammad Khan, Devdutt Sharma, N. Narayanan Prasanth, and S.P. Raja
ABOUT THE COVER The Authority Pathway for Weapon Engagement was integrated in an allied C2 system, demonstrated, and evaluated during an international exercise where one operator controlled multiple heterogeneous unmanned systems in the air, on the ground, at the sea, and under the water. BACKGROUND IMAGE LICENSED BY INGRAM PUBLISHING
Departments & Columns
4 Editorial 6 President’s Message 44 Society News 47 Conference Reports
Mission Statement The mission of the IEEE Systems, Man, and Cybernetics Society is to serve the interests of its members and the community at large by promoting the theory, practice, and interdisciplinary aspects of systems science and engineering, human–machine systems, and cybernetics. It is accomplished through conferences, publications, and other activities that contribute to the professional needs of its members. Digital Object Identifier 10.1109/MSMC.2022.3160369
2
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE A pri l 2022
IEEE Systems, Man, and Cybernetics Magazine EDITOR-IN-CHIEF Haibin Zhu Nipissing University, North Bay, Ontario, Canada [email protected]
ASSOCIATE EDITORS
Okyay Kaynak, Vice President, Organization and Planning Shun-Feng Su, Vice President, Publications Ying (Gina) Tang, Vice President, Finance Vladik Kreinovich, Treasurer
Mali Abdollahian, Australia Mohammad Abdullah-Al-Wadud, Saudi Arabia Choon Ki Ahn, Korea Bernadetta Kwintiana Ane, India Krishna Busawon, UK György EIgner, Hungary Liping Fang, Canada Giancarlo Fortino, Italy Hossam Gaber, Canada Aurona Gerber, South Africa Jason Gu, Canada Abdollah Homaifar, USA Okyay Kaynak, Turkey Kevin Kelly, Ireland Kazuo Kiguchi, Japan Abbas Khosravi, Australia Vladik Kreinovich, USA Wei Lei, China Kovács Levente, Hungary Xiaoou Li, USA Darius Nahavandi, Australia Chris Nemeth, USA Vinod Prasad, India Hong Qiao, China Ferat Sahin, USA Mehrdad Saif, Canada Bahram Shafai, USA Weiming Shen, Canada Liqiong Tang, New Zealand Ying Tan, Australia Yingxu Wang, Canada Margot Weijnen, Netherlands Peter Whitehead, USA Zhao Xingming, China Laurence T. Yang, Canada Qiangfu Zhao, Japan
Tom Gedeon, Secretary
SOCIETY BOARD OF GOVERNORS
Publications Ethics Committee Shun-Feng Su, Chair Imre Rudas Edward Tunstel Vladik Kreinovich Peng Shi Fei-Yue Wang Robert Kozma Ljiljana Trajkovic Haibin Zhu
Executive Committee Sam Kwong, President Imre Rudas, Jr. Past President Edward Tunstel, Sr. Past President Enrique Herrera Viedma, Vice President, Cybernetics Saeid Nahavandi, Vice President, Human–Machine Systems Thomas I. Strasser, Vice President, Systems Science and Engineering Yo-Ping Huang, Vice President, Conferences and Meetings Karen Panetta, Vice President, Membership and Student Activities
Valeria Garai, Asst. Secretary Editors Peng Shi, EIC, IEEE Transactions on Cybernetics Robert Kozma, EIC, IEEE Transactions on Systems, Man, and Cybernetics: Systems Ljiljana Trajkovic, EIC, IEEE Transactions on Human–Machine Systems Bin Hu, EIC, IEEE Transactions on Computational Social Systems Dongrui Wu, EIC, SMC E-Newsletter Industrial Liaison Committee Christopher Nemeth, Chair Sunil Bharitkar Michael Henshaw Yo-Ping Huang Azad Madni Rodney Roberts Organization and Planning Committee Vladimir Marik, Chair Enrique Herrera Viedma Mengchu Zhou Dimitar Filev Robert Woon Ferat Sahin Edward Tunstel Larry Hall Jay Wang Michael Smith C.L. Philip Chen Karen Panetta
History Committee Michael Smith Membership and Student Activities Committee Karen Panetta, Chair György Eigner, Coordinator Christopher Nemeth
Chapter Coordinators Subcommittee Lance Fung, Chair Enrique Herrera-Viedma Imre Rudas Adrian Stoica Maria Pia Fanti Karen Panetta Hideyuki Takagi Ching-Chih Tsai
Lance Fung Robert Kozma Roxanna Pakkar Saeid Nahavandi Okyay Kaynak Tadahiko Murata Ferial El-Hawary Paolo Fiorini Shun-Feng Su Virgil Adumitroaie Peng Shi Ashitey Trebi-Ollennu Hideyuki Takagi
Student Activities Subcommittee Roxanna Pakkar, Chair Bryan Lara Tovar Piril Nergis JuanJuan Li X. Wang
Standards Committee Loi Lei Lai, Chair Henry Chung Chun Sing Lai Wei-jen Lee Thomas Strasser Kit Po Wong Fang Yuan Xu Chaochai Zhang Jizhong Zhu
Young Professionals Subcommittee György Eigner, Chair Ronald Bock Sonia Sharma Xuan Chen Raul Roman Fernando Schramm
Nominations Committee Imre Rudas, Chair C.L. Philip Chen Vladimir Marik Ljiljana Trajkovic Awards Committee Dimitar Filev, Chair Edward Tunstel Laurence Hall Ljiljana Trajkovic Peng Shi Michael H. Smith Vladik Kreinovich
IEEE PERIODICALS MAGAZINES DEPARTMENT 445 Hoes Lane, Piscataway, NJ 08854 USA Peter Stavenick Journals Production Manager Katie Sullivan Senior Manager, Journals Production
Fellows Committee Dimitar Filev, Chair Edward Tunstel, Vice Chair Okyay Kaynak Robert Kozma Vladimir Marik Fei-Yue Wang Sam Kwong Jun Wang
Janet Dudar Senior Art Director Gail A. Schnitzer Associate Art Director Theresa L. Smith Production Coordinator
Electronic Communications Subcommittee Saeid Nahavandi, Chair Syed Salaken, Web Editor Darius Nahavandi, Social Media Mariagrazia Dotoli Patrick Chan Haibin Zhu Ying (Gina) Tang Ferat Sahin György Eigner
Mark David Director, Business Development— Media & Advertising Felicia Spagnoli Advertising Production Manager Peter M. Tuohy Production Director Kevin Lisankie Editorial Services Director Dawn M. Melley Staff Director, Publishing Operations
IEEE SYSTEMS, MAN, AND CYBERNETICS MAGAZINE (ISSN 2333-942X) is published quarterly by the Institute of Electrical and Electronics Engineers, Inc. Headquarters: 3 Park Avenue, 17th Floor, New York, NY 10016-5997 USA, Telephone: +1 212 419 7900. Responsibility for the content rests upon the authors and not upon the IEEE, the Society or its members. IEEE Service Center (for orders, subscriptions, address changes): 445 Hoes Lane, Piscataway, NJ 08855-1331 USA. Telephone: +1 732 981 0060. Subscription rates: Annual subscription rates included in IEEE Systems, Man, and Cybernetics Society member dues. Subscription rates available on request. Copyright and reprint permission: Abstracting is permitted with credit to the source. Libraries are permitted to photocopy beyond the limits of U.S. Copyright law for the private use of patrons 1) those post-1977 articles that carry a code at the bottom of the first page, provided the per-copy fee indicated in the code is paid through the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923 USA; 2) pre-1978 articles without a fee. For other copying, reprint, or republication permission, write Copyrights and Permissions Department, IEEE Service Center, 445 Hoes Lane, Piscataway, NJ 08854. Copyright © 2022 by the Institute of Electrical and Electronics Engineers Inc. All rights reserved.
IEEE prohibits discrimination, harassment, and bullying. For more information, visit http://www .ieee.org/web/aboutus/whatis/policies/ p9-26.html.
Digital Object Identifier 10.1109/MSMC.2022.3160368
Ap ri l 2022
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE
3
Editorial
by Haibin Zhu
Human Brains Plus Abstract Thinking Bring in the Progress of Technologies
T
his is my first editorial for IEEE Systems, Man, and Cybernetics Magazine, and I am honored to take the role as its editor-in-chief. I appreciate the effort of former Editor-in-Chief Prof. Saeid Nahavandi in the editing process of collecting the exciting feature articles for this issue. The first article, “Frontiers of Brain-Inspired Autonomous Systems: How Does Defense R&D Drive the Innovations?” presents new insights into autonomous systems of a top defense scientist of Canada and a group of highly active IEEE fellows in the IEEE Systems, Man, and Cybernetics Society. The authors propose that a brain-inspired intelligent adaptive system (IAS) will bring in fundamental breakthroughs in the cognitive aspect of humans and the incompetence of artificial intelligence. In practice, IASs have led to defense science and technology innovations for trustworthy mission-critical autonomous systems. They deem that these paradigms will empower highly automated systems to think and behave like humans for generating collective intelligence. They believe that IASbased autonomous systems not only fostered the development of a series of novel theories and methodologies, Digital Object Identifier 10.1109/MSMC.2022.3148912 Date of current version: 25 April 2022
4
Haibin Zhu
but also paved unprecedented paths to innovative applications in the defense and general industries. The second article, “Home Energy Management Systems: Operation and Resilience of Heuristics Against Cyberattacks,” is dedicated to a group of researchers from Lappeenranta University of Technology, Finland. They think that the Internet of Things and advanced communication technologies have demonstrated great potential to manage residential energy resources by enabling demand-side management (DSM). Home energy management systems (HEMSs) can provide automatic control for electricity production and usage inside homes using DSM techniques by col-
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE Apri l 2022
lecting information from hardware components. A major challenge of HEMSs is their vulnerability to cyberattacks. A cyberattacker may influence the pricing information of a customer’s HEMS and finally lead to wrong automation outputs. This article introduces demand–response (DR)-based DSM in HEMSs and discusses DR optimization using heuristic algorithms. Moreover, it discusses the potential impacts of cyberattacks and the effectiveness of heuristic algorithms against them. This article also presents open questions for future research work. Interestingly, tensors, a mathematical model, are applied in the other two articles, which use tensors in modeling the related industry applications. The third article, “Tensor-Based Knowledge Fusion and Reasoning for Cyberphysical-Social Systems,” is contributed by a research team at Huazhong University of Science and Technology, China. They think that cyberphysical social systems (CPSSs) integrate human, machine, and information into large-scale automated systems and generate complex heterogeneous big data from multiple sources. Traditionally, knowledge graphs and the Resource Description Framework are used to describe knowledge in designing CPSSs. However, the authors believe that the graph structure does not have flexibility and computability in the theoretical framework, even though it is easily understandable. Therefore, they propose a tensor-based knowledge analysis framework that supports the computability of knowledge graphs. They first employ Boolean tensors to represent heterogeneous knowledge graphs and then present a series of graph tensor operations of high-order knowledge graphs. Furthermore, they perform a tensor 1-mode product
predict the price operation to obtain of Bitcoin, which is the relation path They propose attractive to those tensor, to infer the a tensor-based who like to invest relationsh ip be knowledge analyin this market. As tween any two entistated by the aut ies. F i na l ly, t hey sis framework t hor s, Bit coi n i s d e mo n s t r a t e t he that supports the world’s most practicality and efthe computabiltraded cryptocurfectiveness of the ity of knowledge rency and is highly proposed model popula r a mong by implementing a graphs. cr y ptocurrency case study. investors and minA nother i nterers. However, its volatility makes it esting article, “Bitcoin Price Prea risky investment, which leads to diction in a Distributed Environthe need for accurate and fast pricement Using a Tensor Processing prediction models. These researchUnit,” accomplished by a group of ers propose a Bitcoin price-predicresearchers from Vellore Institute tion model using a long short-term of Technology, India, also applies a memory network in a distributed entensor-based approach to applivironment. A TPU has been used to cations, using a tensor processing provide the distributed environment unit (TPU). They inform us how to
for the model. The r esults show that the TPU-based model performed significantly better than a conventional CPU-based model. I believe that this prediction method provides a good reference for Bitcoin investors and interested researchers. I hope these articles provide inspiration for the research work of our readers. Enjoy the reading! About the Author Haibin Zhu ([email protected]) is a full professor and the coordinator of the Computer Science Program, the founding director of the Collaborative Systems Laboratory, and a member of the University Budget Plan Committee and Arts and Science Executive Committee, Nipissing University, North Bay, Ontario, P1B 8L7, Canada.
IEEE Membership Can Help You Reach Your Personal and Professional Goals
Gain access to the latest IEEE news, publications and digital library. Give and receive personal mentoring. Network locally and globally with IEEE members. And that’s only the beginning. Discover how IEEE can help jumpstart your career.
“Participating in IEEE has developed me as a well-rounded engineer and helped me shine during networking events.” -Likhitha Patha Electrical Engineering Student, IEEE Brand President, Virginia Polytechnic Institute and State University
Visit www.ieee.org/join today.
Ap ri l 2022
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE
5
President’s Message
by Sam Kwong
Expanding Our Focus, Forging Synergy, and Nurturing Talent
D
ear Members of the IEEE Systems, Man, and Cybernetics Society (SMCS), I wish you a happy, fruitful and prosperous 2022. It is my great honor and privilege to serve you all as 2022–2023 president of the SMCS. This is a huge privilege and a tremendous responsibility, which I can only hope to fulfill. I am highly grateful for the trust in me and the many messages of encouragement and support that I have received since I have been elected. As president of the SMCS, I hope to use my experience, dedication, and vision to serve our community in the upcoming year. I will dedicate my best efforts to serve the SMCS in key areas, such as conducting high-quality technical activities and expanding our focus on socially valuable niche areas, identifying means to monitor the quality of SMCS-sponsored technical committees in cybernetics, working with vice president (VP) membership to recruit and engage w ith new member s, empower i ng conference quality through enabling Technical Committee chairs and members to work with conference organizers, promoting synergy with industry and creating cutting-edge
Digital Object Identifier 10.1109/MSMC.2022.3148917 Date of current version: 25 April 2022
6
Sam Kwong
technological knowledge to support social and economic progress, and nurturing talent to be the future leaders of our society. I want to thank all of our Society members and volunteers for their continuous support, service, and dedication to the SMCS. I am truly grateful for your contribution of expertise and valuable time for our Society and our members. In particular, I would also like to thank the members of the SMCS Executive Committee (ExCom) and Board of Governors (BoG) for their help and support throughout the past two years and for their dedicated service. In particular, I would like to thank our outgoing ExCom and BoG members. My sincere thanks go to Dimitar Feliv for his extraordinary contributions to the Society as president of the
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE Apri l 2022
SMCS in 2016–2017. His achievements have been recognized by the IEEE 2008 Norbert Wiener Award and the IEEE 2015 Computational Intelligence Pioneer’s Award. I would like to thank Junior Past President Imre Rudas for his leadership over the last two years amid unprecedented challenges, particularly the disruptions brought by the COVID-19 pandemic. Although there were no more in-person ExCom and BoG meetings after the 2020 meeting in Mexico, the BoG participated in our Society more actively through virtual meetings under his leadership. Even more impressive was the introduction of Associate VP positions and the Executive Assistant position. Thanks to Vladimir Marik for his excellent contributions to the Society. Marik has served as the VP of Organization and Planning. He developed and regularly evaluated our Operational Plans and Strategic Plan in 2022–2027 and consolidated the Constitution and Bylaws in cooperation with the IEEE Headquarters and others. My appreciation also goes to Andreas Nürnberger for his essential contributions to conferences and meetings. With his hard work and those of our committee members and conference organizers, many conferences are reviewed, approved, and conducted successfully every year. Thanks to Feiyue Wang, the VP of Human–Machine Systems, for his vital help to technical committees’ chairmen in organizing conferences, workshops, seminars, and special sessions, to further the technical excellence of the Society. Finally, my heartiest thank you to Peng Shi, VP of Publications, who has recently moved on to the greener pastures of Editor-in-Chief of IEEE Transactions on Cybernetics.
Finally, I would like to thank all of the outgoing BoG members, Saeid Nahavandi, Yo-Ping Huang, Tom Gedeon, Okyay Kaynak, and Adrian Stoica, for their valuable contributions. Our achievements would not have been conceivable without them, and I am happy to say that many of them will continue to take up more critical positions within the Society. I would like to extend my congratulations to our new ExCom members: Saeid Nahavandi, VP Human; Okyay Kaynak, VP Organization and Planning; and Enrique Herrera-Viedma, VP Cybernetics. I welcome Imre Rudas and Eddie Tunstel to their new roles as Junior and Past Presidents of the Society, respectively. Their advice is valuable to the Society for
its success and smooth operations. I look forward to working with you all. Congratulations to our newly elected members: Plamen Parvanov Angelov, Yaoping Hu, David Mendonca, Ching-Chih Tsai, and Haibin Zhu. I look forward to working with all of you in the coming two years. Finally, I wish you all an excellent 2022. In the upcoming year, we may continue to face the challenges of the COVID-19 pandemic. However, I am humbled by your dedication to our Society despite these difficult circumstances. You have brought diverse experiences and formidable capabilities to our Society and, more important, your passion and commitment to the SMCS. With all your support, I am confident that we can
move from strength to strength in the year ahead. About the Author Sam Kwong earned his B.Sc. degree from the State University of New York at Buffalo in 1983; his M.A.Sc. degree in electrical engineering from the University of Waterloo, Ontario, Canada, in 1985; and his Ph.D. degree from the University of Hagen, Germany, in 1996. He is a professor in the Department of Computer Science, City University of Hong Kong, Kowloon, Hong Kong. He is the associate editor of IEEE Transactions on Industrial Informatics, IEEE Transactions on Industrial Electronics, Evolutionary Computation, and Journal of Information Science. He is a Fellow of IEEE.
®
Where technology and philanthropy intersect Together, we deliver opportunity, innovation and impact across the globe.
DONATE NOW Make a donation: ieeefoundation.org/donate
Ap ri l 2022
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE
7
Frontiers of Brain-Inspired Autonomous Systems How Does Defense R&D Drive the Innovations?
by Ming Hou, Yingxu Wang, Ljiljana Trajkovic, Konstantinos N. Plataniotis, Sam Kwong, MengChu Zhou, Edward Tunstel, Imre J. Rudas, Janusz Kacprzyk, and Henry Leung
©SHUTTERSTOCK.COM/KENTOH
Digital Object Identifier 10.1109/MSMC.2021.3136983 Date of current version: 25 April 2022
8
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE Apri l 2022
©2022 Canadian Crown Copyright
A
brain-inspired intelligent adaptive system (IAS) framework is developed toward fundamental breakthroughs in the cognitive bottleneck of humans and the incompetence of artificial intelligence (AI) under indeterministic conditions or with insufficient data. IASs have led to defense science and technology innovations for interaction-centered design (ICD) methodologies, human–autonomy symbiosis initiatives, and a trust framework synergizing key strategies on intention, measurability, performance, adaptivity, communication, transparency, and security (IMPACTS) for trustworthy mission-critical autonomous systems. These paradigms of emerging technologies empower highly automated systems to think and behave like humans for generating collective intelligence. IAS-based technologies have not only fostered the development of a series of novel theories and methodologies, such as brain-inspired systems and the ICD approach, but also have paved unprecedented paths to innovative applications in the defense and general industries. Background It is recognized that many fundamental theories and innovative technologies have been discovered or triggered by defense R&D, including modern computers, the Internet, and autonomous systems. AI has become ubiquitous and capable of rendering autonomous reasoning and decisions. It continuously changes the roles and responsibilities of human functions in human–machine symbiotic partnership. Human roles are becoming increasingly supervisory in nature, and more contextual decisions are applied to technology [1]. This trend poses important questions about the limitations, liabilities, risks, privacy, ethics, and trust associated with increasing autonomous decision-making capabilities in safety and mission-critical applications, such as self-driving vehicles, home care and surgical robots, Industry 4.0 smart manufacturing systems, or remotely piloted combat drones [2]–[6]. Data-driven AI algorithms require big data to train that may not be suitable for many real-time applications. When facing insufficient data, incomplete information, indeterministic conditions, or inex haustive solutions for u ncer ta in
actions, data-regression AI is unable to provide timely support in decisions regarding the what, where, when, who, why, and how associated with operational situations due to the absence of autonomous decision-making theories [7], [8]. With the increased contextual complexity and behavioral opacity of AI, it is even more challenging for humans to maintain sufficient situational awareness (SA) and safe responsibility transfer during the transition from “on-the-loop” to “in-theloop” when AI and autonomy are coupled closely with humans [9]. The two fatal crashes of the Boeing 737 Max in 2018 and 2019 are typical examples of a failed responsibility transfer between an AI-enabled autonomous function and pilots due to a faulty assumption about human cognitive capacities and disregard for design principles when developing new technologies based on outdated designs created a few decades ago [10]. The investigation report on another airplane incident due to a machine failure, the emergency landing of US Airways 1549 in the Hudson River in 2009, revealed that the pilot needed sufficient resources (time and attention) to process information, assess the situation, make the right decisions, and take over the control when the machine (aircraft) failed [9], [11]. These examples reiterate that the design of these emerging technologies must seriously consider the capability limitations of both human and machine intelligence from the onset rather than as an afterthought. Further, networked systems with associated interconnectivity and interdependence accelerate the spiking global complexity. Thus, it is paramount to realize the imperative needs of guidance for understanding and mitigating the risks during the design, development, validation, certification, and exploitation processes for these sociotechnical systems [3]. In fact, the entire spectrum of autonomous systems demands innovative technological solutions, enduring strategies, science-based design methodologies, and evidence-based process standards to ensure that these emerging technologies can be trusted and employed safely, effectively, reliably, legally, and ethically before AI or autonomy is integrated more widely into our systems, operations, a nd society [12]–[15]. This article presents the latest defense R&D-driven autonomous systems and a novel brain-inspired IAS as a systematic solution to the aforementioned issues. An associated coherent body of ICD methodologies and an IMPACTS trust framework are elucidated as enduring strategies and structured guidance for designing trustworthy IASs where critical decisions are made by AI, humans, or both. Real-world brain-inspired IAS examples are demonstrated as strategy and paradigm validation studies. Future directions are suggested for broa der ICD - a nd I M PACTS -relat ed appl icat ion,
Ap ri l 2022
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE
9
doctrine, and process endeavors in an IAS design, development, validation, certification, and exploitation lifecycle. The Emergence of Brain-Inspired IASs It is commonly assumed that autonomous systems toward general AI may be achieved when brain-inspired cognitive capabilities for decision making are available to exhibit human-like behaviors of both intelligence and adaptability. Recent theoretical advances and technological convergence in cybernetics, system science, cognitive informatics, and augmented cognition offer possibilities of a brain-inspired and knowledge-based IAS that harnesses leap-ahead technologies, such as the next generation of computing (quantum computing and cognitive computing) [9], [16]–[18]. Thus, a machine is able to “think” like humans to improve decision making and generate anticipatory intelligence by mimicking the neurological processes of the human brain. With the collective human–machine intelligence, an IAS is able to effectively draw inferences from data, including those that are incomplete, uncertain, or ambiguous, and learn from experience and users’ interaction responses. An IAS is capable of changing its behavior in real time as a function of a user cognitive state and the status of the task, machine, and world (working environment). By optimizing human–machine interactions, an IAS may intelligently adapt to the capabilities, capacities, limitations, needs, and demands of a user and machine to attain and maintain safety, trust, effectiveness, and efficiency when
Interaction Model
Knowledge Model
humans perform various cognitively challenging tasks in complex and dynamic environments [9]. The Architectural Framework of IASs To achieve IAS objectives for its broad range of applications, a system needs to exhibit at least five fundamental characteristics: 1) tracking goals, plans, or intents, and the progress toward them; 2) monitoring and inferring the internal state of a user (behavioral, physical, cognitive, and emotional); 3) monitoring and inferring the external status of the world (environmental conditions, entities, and domain constraints); 4) monitoring the effects of machine status, automation, advice, and adaptation on user and world status (closed-loop feedback); and 5) customizing its human–machine interface (HMI), including a brain– machine interface to handle the interactions and trust relations between a user and a machine. To manifest these characteristics, a conceptual architectural framework has been developed as a basic IAS anatomy with critical components common to knowledge-based IASs [9]. As illustrated in Figure 1, a generic IAS includes four modules pertaining to situation assessment, user state assessment, adaptation engine, and HMI. There are several knowledge models to support each of these modules. Situation Assessment This module is concerned with the assessment of the “situation” and comprises functionality relating to the real-time analysis of the activities (required to achieve a specific goal), automation, and decision support. The module
Task Model, Machine Model, World Model
Situation Assessment Compare HMI
Adaptation Engine
User State Assessment Autonomous Intelligent Behavior Generation
Trust Model
User Model (Behavioral, Contextual, Neurophysiological, and Psychophysiological States)
Figure 1. The architectural framework of a generic IAS with the four modules and their supporting models. 10
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE Apri l 2022
(presenting only critical informamonitors and tracks the current tion in the right format and at the progress toward a specific activiHuman roles right time during periods of excesty, goal, or status through data s i vely h i g h -work lo a d le vel s sensing and fusion from internal are becoming through the HMI). A knowledge (machine status) and external increasingly model supports this module with data sources by using task, goal, supervisory in nature, its abstract representations of the and situational knowledge. The application domain, task, user, knowledge is used for intelligence and more contextual machine, and world. It provides generation and the adaptation decisions are applied baseline conditions for the comengine to decide appropriate parisons between the expected strategies to assist a user through to technology. outcomes and the current status decision support and adaptive of all of these variables. Differencautomation or by adapting the es or deficiencies are used to information presented to the user drive the intelligent adaptation through the HMI. process to optimize human–machine interactions and Underpinning this module are task, machine, and world assist in achieving overall system goals. models. A task model contains knowledge pertaining to the tasks that a user is expected to perform and is represented as an organization of actions, goals, and plans. A HMI machine model includes knowledge related to the machine This module monitors, updates (the models), and commuitself, its abilities, and the means of assistance to support a nicates (with the machine) real-time user behavioral, neuuser, including advice, automation, and interface adaptarophysiological, and psychophysiological changes such tion. A world model defines the external world according that the system may assess a user’s state of overall cognito the objects that exist in the working environment, their tive resources, including attention, engagement, emotion, properties, and the rules that govern them. and workload. Two models supporting this module are the interaction model and trust model. An interaction model includes knowledge related to the mode of communicaUser State Assessment tions and interactions between a user and machine as well This module provides information and knowledge about as among system components. A trust model contains a user’s behavioral, contextual, neurophysiological, and knowledge related to optimal states of variables affecting psychophysiological states within the context of a spetrust between a user and a machine. cific work activity. A supporting user model incorpoThrough real-time data collection about human– rates the knowledge, skills, or behaviors that embody a machine interactions, the changes of the user, task, specific user performing a specific task based on the machine, and world states and those trust factors can be mechanism of human cognition, control abilities, and computed (computational interaction for anticipatory intelcommunications. Through real-time analysis of a user’s ligence). The computational results drive the adaptation interactions, the module updates the system knowledge engine to optimize human interactions with the machine about the modeled state of a user’s attention, engagement through intelligent adaptation of machine behavior and with the tasks, ongoing cognition (visual and verbal proinformation presentation. The computational results also cessing load), emotion, intention, performance, and update the trust levels and SA for a user and machine about competency. The knowledge provides a basis for the the current system status such that decisions can be made adaptation engine, which computes and compares the for further actions to maintain, repair, or regain the desired current user states with the built-in knowledge model of trust (trust calibration and assurance). the user to assess the user’s deficiencies (computational Overall, the four modules operate within the context interaction for anticipatory intelligence). Then, the in of a closed-loop system. In each module, a closed-feedtelligent adaptation of the user assistance may be back loop resamples user state and situation assessment autonomously triggered to enhance and mitigate human to update all knowledge models following the adaptation information processing and decision-making capabilities of the machine or HMI. Thus, an IAS may adjust the level and limitations. of adaptation such that optimal user states (attention, engagement, performance, and workload) and trust are Adaptation Engine attained, maintained, and assured. This conceptual This module employs high-level knowledge outputs from framework provides a complete snapshot of the user, the user state and situation assessment modules to genermachine, task, and the environment with which they ate autonomous intelligent behaviors. It seeks to maxiinteract. This is crucial to the design, development, verifimize the system performance with its functions to cation, validation, certification, and exploitation of comprioritize and optimize task allocations (to either the user plex brain-inspired IASs. or the machine or both) and information management
Ap ri l 2022
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE
11
ICD for IAS Development Systems designers should be aware of user requirements and preferences when designing automation and interface as the two basic components of a human–machine system. Conventional automation and interface are designed following a technology-centered strategy based on task models, as illustrated in Figure 2. To reduce user workload and increase task efficiency and productivity, designers use a task model to preplan and predesign automation functions with an understanding of user requirements and preferences. With the emphasis on how technology advances may help allocate additional tasks to automation, technology acts as an assistant to a user. Meanwhile, the user does not take over the automation’s tasks. Automation capability is derived from the leftover and compensatory principles. Basically, humans need to compensate with the functions that have not been automated or that could not be automated because of the machine limitations [19]. When more tasks are allocated to automation, humans have difficulty in addressing performance issues, such as loss of SA (out of the loop), loss of skills, overtrust (complacency), or undertrust (skepticism) [9], [20]–[22]. Thus, knowledge of a task model has been extended to include a user model of how and what humans are doing such that the machine may provide flexible support. This advanced the design strategy from a technology-centered design approach to a user-centered design (UCD) or human-centered design (HCD) approach in the 1980s with the focus shift from technological capabilities onto human needs [23], [24]. The goal of HCD is to create designs (of products, services, workspaces, systems, procedures, organizations)
that take into account the needs, capabilities, and limitations of those who are using or being impacted by the design for users’ acceptance [24]. The need to assist humans in a flexible fashion has subsequently fostered the development of adaptive or adaptable automation as well as adaptive or intelligent interface technologies [9], as shown in Figure 2. However, the human is only one of the many attributes of a broader human–machine system. The design should emphasize a system as a whole. Vicente [25] has posited that a UCD approach is not always ideal, arguing that a systems design perspective is more advantageous for correspondence domain applications (safety and mission-critical systems) [9]. For example, in the aviation, process control, and medical fields as well as in warfare, a design flaw in a medical instrument or weapon system can have lethal and expensive consequences. The catastrophic disaster at the Three Mile Island nuclear power plant in Harrisburg, Pennsylvania, where a meltdown occurred in 1979, is a typical example in which the design disregarded appropriate design principles for correspondence domain applications [9], [26]. An issue with HCD has been further identified as local optimization that fails to consider the big picture and systems perspective [27]. Norman [27] then recommended modifications of an insufficient HCD approach to address issues for correspondence domain applications with complex sociotechnical systems. As machines replace humans in a variety of tasks and slowly turn into independent entities, these issues regarding human–machine interactions come to the forefront [28]. Sheridan [1] suggests that, as the frontiers between
User Requirements/Preferences Leftover/Compensatory Principle
Conventional Interface
Adaptive Interface/ Intelligent Interface
Intelligent Adaptive Interface
Task Model Technology-Centered Design 1950s Task + User Models UCD/HCD 1980s Task + System (User + Machine) + World Models ICD 2010s
Static Automation
Adaptive Automation/ Adaptable Automation
Intelligent Adaptive Automation
Intelligent Adaptive System Figure 2. The evolution of a design strategy for interface and automation technologies as two critical
components of an IAS from technology-centered design to user-centered design (UCD) or human-centered design (HCD), and then to ICD principles.
12
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE Apri l 2022
intelligent adaptive interface and automation and humans blur, it intelligent adaptive automation becomes “increasingly critical” Shared responsibility into a hybrid system that features that automation designers realize state-of-the-art automation and that they are building not only exhibits functional interface technologies. Figure 2 technology, but also relationships. integration of illustrates the parallel evolution of From a system-of-systems pera design strategy and principles for spective, UCD or HCD is no longer human and machine interface and automation technolosufficient to address broader intelligence gies from technology-centered human–machine interaction and design to UCD/HCD and then to relational issues, especially for within human– ICD. It also demonstrates the condomain applications with sociomachine symbiotic sistencies in their evolution and technical complexity [12], [25], partnership, which is their eventual amalgamation into [27]. Thus, the knowledge of task the IAS. and user models has been broada key IAS human-like ened to include models of the characteristic. intended machine and the world ICD Impacts on IAS Capability (i.e., the working environment of and Standard Development an anticipated system). The aim is Over the past decade, the IAS to foster the development of both framework, ICD approach, and a intelligent and adaptive automation and interface techset of associated analytical and development methodolonologies guided by an ICD methodology [9], [29], as gies were applied to develop a variety of defense capabilishown in Figure 2. ties for the Canadian Armed Forces (CAF). One example A primary goal of the ICD approach is to optimize is the first Canadian Intelligent Tutoring System (ITS) for human–machine interactions for IASs based on their joint Improvised Explosive Device Disposal (IEDD) operator capabilities, strengths, and limitations to maximize overall training. An innovative intelligent adaptive learning syssystem performance, ensure safety, and enable trust. This tem architecture was created based on an IAS framework requires the machine to be equipped with human-like to guide the development of the ITS that enabled IEDD intelligence and behaviors so that the issue of human cogtrainees to interact dynamically with training scenarios nitive bottlenecks (limitations in attention, memory, learnand receive real-time feedback on their questioning skill ing, comprehension, visualization abilities, and decision acquisition. This resulted in an increased course success making) may be effectively addressed. With the humanrate from 60 to 94% with reduced cost [31], [32]. Because like cognitive capabilities of perceiving, reasoning, interof its novel IAS concepts based on the ICD approach, this preting, and predicting the current and future status of a ITS technology has been filed for a patent application in user, task, machine, and environment, the machine can Canada and the United States and exploited by CAE Inc. predict human activities, awareness, intention, resources, to create a commercial intelligent tutoring program for and performance. It may then share responsibilities with aviation training [33]. its human partner and proactively assist in timely decision The ICD approach has also been applied for the developmaking on task execution, automation adaptation and ment of the first Canadian Unmanned Aircraft System management strategy, system behaviors, and transfer of (UAS) Command and Control (C2) center that consists of a control and authority. Ground Control Station (GCS) for supporting a Canadian Shared responsibility exhibits functional integration of major capital UAS acquisition project. A series of empirical human and machine intelligence within human–machine studies conducted using this IAS capability has provided symbiotic partnership, which is a key IAS human-like charscientific evidence that informs the development of requireacteristic [9], [29], [30]. Functional integration is extremely ments for the project Request for Proposal, UAS GCS Workimportant in emergencies because it can create robust sysspace Optimization and Airworthiness Certification, and tems to handle unexpected events. For instance, safety Operator Training Technology and Strategy [34], [35]. redundancies should be built into aircraft control systems Another GCS has been deployed as a new trainer for the to allow an alternate course of action if a key component joint UAS operator training for Canadian Army, Navy, and fails. If this strategy had been followed in the Boeing 737 Special Operations Forces. The ICD approach has also Max design, the two catastrophic accidents in 2018 and guided the development of the Canadian Army Statement 2019 could have been avoided even though the AI-enabled of Requirements for Micro UASs [36]. Thales Inc. has adoptfunction failed. ed related cognitive aspects of an IAS in its AI program on The ICD methodology satisfies IAS design requirements Human Sensing Technology for intelligent adaptation and mitigates potential risks through detailed and comprebased on operator intent and workload [30], [35]. hensive knowledge of a user, task, machine, and environThe ICD paradigm has been recognized by NATO, ment. An IAS is essentially the unified evolution of an which adopts the systematic and structured IAS
Ap ri l 2022
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE
13
framework and ICD approach as an enduring strategy with guiding principles for the development of the NATO standards “Guidance on Sense and Avoid for UAS” and “Human Systems Integration Guidance for UAS” to support the efforts of integrating UASs into the nonsegregated civilian airspace [37], [38]. The related IAS design and development processes have been regarded as validated best practices and advocated through the invited NATO Lecture Series on UAS technical challenges, concepts of operations, and regulatory issues [39]–[41].
An effective IAS should enable machine adaptivity through dynamic interactions between humans and machines and, thus, foster the development, maintenance, and assurance of mutual trust.
Impacts: A Trust Model for Human–Machine Symbiotic Partnership An IAS requires active human–machine interactivity at the highest level where both human and machine support each other proactively. It means that system control (authority) and responsibilities can be transferred safely between the partners whenever necessary. Thus, a trusted relationship needs to be built and maintained through dynamic interactions. Trust has then been identified as a key element and a “fundamental enabler” of a collaborative IAS decisionmaking capability [42]. Trust is a psychosocial relationship between entities or agents capable of acting. It is commonly understood as a cognitive process and a relational mediator for interactions among humans, humans and organizations, and human–machine teaming. A variety of models examines trust factors and describes the development of trust in
Intention Measurability Performance Adaptivity Communication Transparency Security
Trust Figure 3. An IMPACTS trust model to exhibit the seven
integrity characteristics of an IAS based on ICD methodology.
14
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE Apri l 2022
automation, cloud computing, blockchains, and Industr y 4.0 smart manufacturing [4], [43]–[45]. However, these tr ust models, although comprehensive, may not consider dimensions related to the ever-increasing AI capabilities and their impacts on the aggregated decision-ma k i ng powers that reside in IASs.
The IMPACTS Trust Model As machines evolve into highly autonomous functions of IASs with greater AI decision-making capabilities, the studies in [42] and [46] have suggested that the human–machine trust relationship shou ld mor e clo s ely m i r ror human-to-human trust to reflect the dynamic and complex nature of human–autonomy teaming. Trust is a function of capability and integrity for human relationships [42]. Human trust in technologies cannot exist without either of these two variables. Technologically, IASs with AI-enabled autonomous functions are more capable than their human partners in certain areas. To build up human-like integrity, the IMPACTS model has been developed for IASs to exhibit seven characteristics to gain actual trust from humans [42]. As illustrated in Figure 3, the defining integrity characteristics of the model are shared intention, performance measurability, predictable and reliable performance, context adaptivity, bidirectional communication, optimal transparency, and protective security. ◆◆ Intention: A collaborative partnership should have the desire for mutual support. The chosen desires are defined as intentions to which a machine commits efforts and resources for achieving the goals to help its human partners. The machine should know what humans are trying to achieve such that it may pursue actions that achieve its intention to help. Meanwhile, understanding a machine’s supportive intention toward a common goal serves as a starting point for humans to trust their machine partner. Thus, the design of an IAS should allow the demonstration of shared intentions of humans and machines through their social interactions. ◆◆ Measurability: The development of trust is a dynamic process and should not be undermined by a single instant. One may not trust words and may even question actions, but one should not doubt patterns. The same human–human relational philosophy can be applied to the human–machine partnership, where an AI entity may be complex and opaque. AI entities should be measurable such that their behaviors may be obser ved, their actions measured, and their
behavioral patterns analyzed to gauge intentions. Thus, trust is developed through observable behaviors, measurable actions, or analyzable patterns through these types of computational interactions. The computational results and implications may then be judged as supportive or not. ◆◆ Performance: Trust should be built, earned, maintained, and assured. The development of trust results from the reliable, consistent, and predictable performance of a partner over time [44]. In fact, performance is identified as the primary contributor for establishing trust for robots and significant for the establishment of trust for automation [47]. Performance entails a variety of attributes. For a machine to gain trust from its human partner, it must demonstrate performance that is valid (exhibiting as intended), reliable (being consistent over time), dependable (having few errors), and predictable (meeting human expectations). ◆◆ Adaptivity: AI entities need to be capable of learning and understanding their human partners’ intentions and also the machine status and changes in the tasking environment, monitoring the human cognitive workload and performance, guarding the human resources and time, and changing their course of action to work with humans to achieve the team’s common goals. An AI entity that exhibits these adaptive and intelligent characteristics may then be a trusted partner to build a truly collaborative human–AI symbiotic partnership. An effective IAS should enable machine adaptivity through dynamic interactions between humans and machines and, thus, foster the development, maintenance, and assurance of mutual trust. ◆◆ Communication: Communication is a key to team success and needs to be bidirectional for humans and machines to learn and understand each other as partners. An effective and trustworthy IAS should enable AI entities and their human partners to clearly, fairly, and directly exhibit and justify their intentions, actions, and desired end states and how they may help each other reach their common goals. These are critical characteristics that machines must exhibit to build up human confidence and trust. An IAS also needs to be flexible and offer the types of feedback that the human partner would like to receive and accept such that effective communications occur at the right time, in the right format, through the right channel, and to the right recipient. If an IAS could facilitate such effective communications, an appropriate trust partnership would be enabled, maintained, and assured constantly. ◆◆ Transparency: It has often been said that AI runs in a black box because it works in a fashion that humans do not fully understand and that they have no way of validating regarding its intentions. The opacity of AI is problematic for trust development, maintenance, and assurance. Opacity has remained acceptable when the
machine is reliable and is designed for simple tasks. Yet, when AI is expected to perform complex tasks, involving multifaceted decisions in uncertain situations alongside humans with potentially vital consequences, transparency is crucial for humans to ascertain that AI entities’ goals and methods for achieving those goals are aligned with shared intentions. Transparency can be facilitated and optimized through an explainable HMI that communicates real-time information to humans and machines regarding their intentions, goals, reasons, decisions, actions, and expected outcomes. ◆◆ Security: IAS capabilities are continuously enhanced by the growth of AI-enabled functionalities; however, it naturally increases the system complexity. In engineering, complexity generally translates into uncertainty and risk, and this generalization applies to the design of IASs. The design should protect the system from accidents or deliberate threats (e.g., a cyberattack) and, thus, build human trust. A secured system must behave as designed and implemented following rules or laws even when it is under attack; otherwise, it cannot gain or assure trust. This insight requires designers and organizations to build confidence in IAS technologies by providing goal-directed explanations of security measures in place (i.e., at the right level of detail) to protect and ensure system safety and performance. According to the IMPACTS model, trust is a vertex and a crucial ingredient for collaborative human–machine symbiotic partnership. It is the careful balance on which healthy relationships grow and are maintained and assured among partners when considering physical, intellectual, emotional, ethical, moral, relational, and even spiritual aspects of human beings. For technologies to be truly trustworthy, consistent, predictable, and reliable and to demonstrate human-like integrity with shared intentions through their adaptive behaviors and measurable performance, transparent communications, and secured protection are indeed “IMPACTS” that only humans can make with their AI partners. Researchers and practitioners developing IASs must carefully design them to inspire confidence and build trust. The IMPACTS model is a conceptually practical tool to guide the design and development of collaborative and trusted human–AI symbiotic partnerships. IMPACTS for Trustworthy IAS Development and Acceptance To address complex, lengthy, and error-prone target engagement processes, the IMPACTS model has been applied for the development of an IAS called the Authority Pathway for Weapon Engagement (APWE) as an intelligent decision support system [48]. The APWE automatically streamlines engagement processes and enables operators to visualize the dynamic engagement status intuitively through its intelligent, adaptive HMI, thereby reducing engagement times and errors while enhancing operators’ SA. Ap ri l 2022
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE
15
One of the critical computational interaction capabilities of the APWE is its function that automatically generates system intelligence with gathered information and knowledge about operators’ states, the tempo of mission tasks, assets, and entities in the battlespace. For example, it calculates, verifies, and advises whether a potential strike (i.e., lethal versus nonlethal) follows the appropriate legal and ethical policies (e.g., laws of armed conflict, rules of engagement, and standard operating procedures) before decisions are made by human authorities. The APWE is an integral part of a joint C2 system that consists of a variety of highly automated emerging and disruptive technologies (EDT) from four nations [49]. The utility, effectiveness, and interoperability of these EDTs, including the APWE, were assessed
in a large-scale and complex international exercise within a context of a single operator controlling multiple heterogeneous unmanned systems in the air, on the ground, at the sea, and under the water, as illustrated in Figure 4. The evaluation results have revealed that military participants identify the APWE as one of the top three strengths of the joint C2 system and a significant contributor to the success of the exercise. Its implementation has been considered as “exemplary, with major enhancements” because it “takes a lot of stress away from the operator.” More important, it is the “most trustworthy” disruptive technology of the entire joint C2 system because it provides increased SA with reduced workload and potential human errors, according to [42] and [49].
Figure 4. The APWE was integrated in an allied C2 system, demonstrated, and evaluated during an
international exercise where one operator controlled multiple heterogeneous unmanned systems in the air, on the ground, at the sea, and under the water. The IAS framework, ICD design methodology, and IMPACTS trust model have guided the design and development of an AI-enabled decision-aid APWE, to address the complex, lengthy, and error-prone target engagement process. The weapon cannot be released unless the APWE completes all steps successfully based on built-in rules of engagement, international laws of armed conflicts, and standard operating procedures. BF: blue force; CDE: collateral damage estimation; Civ: civilian; GBU: guided bomb unit; PID: positive identification.
16
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE Apri l 2022
Given the dynamic, complex, and cognitively challenging nature of many mission-critical military systems with AI-enabled autonomous functions, the IAS framework, ICD approach, and IMPACTS model have been employed to support a series of CAF capability and concept development and evaluation activities. These activities provided various venues to validate the IAS paradigms for broader defense and civil applications, such as autonomous transportation, home care and surgical robots, and Industry 4.0 smart manufacturing [3]–[5]. Meanwhile, the innovative IAS framework and ICD approach are praised by academic experts, industrial practitioners, government authorities, and users from operational communities for their novelty and trend-setting initiatives of human–AI symbiotic partnership. The ICD principles are referred to as “a must read (consideration) for any serious professional in academia, government, or industry” and an excellent guide to the design of “21st century human-computer symbiosis technologies” [9]. They are noted for setting the agenda for the coming years as human factors practitioners grapple with the demands that IASs will make on its operators and outlining how collaboration and partnership between human and AI can be achieved through ICD, according to [50]. IAS broad acceptance, significant impact, and exceptional contribution have been recognized by the Department of National Defence, Canada, and the Professional Institute of the Public Service of Canada with the prestigious Science and Technology Excellence Award and the President’s Achievement Award in 2020 and 2021, respectively. Future Work in Symbiotic Human–Robot Teaming and System Verification Regimes One of the main strengths of the ICD approach is its systematic and structured process with stakeholder involvement and identification of system requirements and critical decisions with associated tasks for sociotechnical systems in correspondence domain applications. It addresses a key challenge of system design: the need to elucidate, develop, and validate operational requirements that are obscured by the complexity of human–machine interactions and of the system itself. This has been demonstrated with the validation studies on a number of military systems discussed previously and adopted by NATO standards. The next step is to systematically integrate the ICD paradigm into defense policies or acquisition processes and applying the systems engineering approach for defining, designing, developing, testing, acquiring, and employing IAS technologies. The IMPACTS trust model is a practical, conceptual component supporting the HMI module in the IAS architecture, as shown in Figure 1. It is an integral part of the ICD approach and is being exploited as a systems engineering analysis and design tool for a trust-management system (TMS) in a context of soldier–robot teaming (SRT). Meanwhile, a TMS-related mathematical matrix is also being developed to measure trust in real time during
human interactions with various autonomous systems. These mechanisms can then be implemented and integrated into SRT technologies for a series of military exercises. Once validated through these operational trials, these paradigms should make significant impacts on the systematic design and validation methodology for enabling trust. The focus on human–machine trust relationships thus far has been mostly on the human’s trust in the machine. However, to fully consider the safety property of a human– machine symbiotic partnership, additional trust relationships of machine-to-machine and machine-to-human partnership (i.e., does the machine trust the human’s decision making or judgment?) must be considered. A potential risk might be a machine’s blind trust in human decisions without knowing the decisions made under the impacts of logical and emotional trust attributes, such as stress or bias [42]. From a computation perspective, machine learning may be rigorous without introducing bias if its algorithm is trustworthy. That is, cognitive and behavioral bias is often caused by interference inconsistency between machines and humans. Hence, overcoming bias potentially requires bidirectional communication and a comprehensive mechanism through overlaid interactions. These mutual trust relationships are therefore suited for representing a more comprehensive and complete trust partnership. Thus, additional studies need to be conducted to understand and develop strategies for managing them simultaneously in real time. For example, how does a brain-inspired machine learn to trust or assess confidence in human judgment or decisions? Or what should be done when it does not trust the human while the human does not trust the machine? Trust mediation has yet to be addressed and should be studied. Further, legal and ethical issues concerning the use of highly automated systems have been identified in [2] and [12], including a sensitive topic when considering safety and mission-critical weapon systems [6]. These issues include the possibility that a system with autonomous functions may purposely and deliberately withhold information concerning a system failure, malfunction, or error. The question has been raised as to whether certain trust repair strategies are ethical, and more work is needed to address these issues. One area is to integrate both a TMS for a measure of trust (MoT) and an ethical design review (EDR) in verification regimes, such as formal systems engineering processes and industry production standards for autonomous transportation, home care and surgical robots, and Industry 4.0 smart manufacturing [15], [51]. This may include analyses of trust and ethics requirements during a system design process and then quantitative measures of the tradeoffs (processing time, memory use, performance, potential misuse, bias, and so on) during a test and evaluation process. Accordingly, a “trust certificate” and/or “ethics certificate” can be issued if test results are satisfactory through Ap ri l 2022
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE
17
Brain Institute at the University of Calgary, Calgary, Alberta, T3A 3H7, A secured system Canada. He has held visiting profesmust behave as sorships at the University of Oxford (1995/2019–2022), Stanford Universidesigned and ty (2008/2016), the University of Caliimplemented fornia, Berkeley (2008), and the Massachusetts Institute of Technology following rules or laws (2012). He is the founding president even when it is under of the International Institute of Cogattack; otherwise, nitive Informatics/Computing (I2CICC). He has published 600+ it cannot gain or articles and 36 books and been recassure trust. ognized by Google Scholar as the World’s top 1 in Software Science, Cognitive Robots, and Knowledge Science. He is a Fellow of IEEE, British Computer Society, I2CICC, the Asia-Pacific Artificial Conclusion Intelligence Association, and World Innovation Foundation. At the frontier of brain-inspired autonomous systems, IASs Ljiljana Trajkovic ([email protected]) earned her Dipl. Ing. have emerged to exhibit collective intelligence enabled by degree from the University of Pristina, Yugoslavia, her M.Sc. optimized human–machine interactions based on their joint degrees in electrical engineering and computer engineering capabilities and strengths to achieve shared goals. The IAS from Syracuse University, and her Ph.D. degree in electrical technology enables trustworthy solutions of human cogniengineering from the University of California at Los Angetive bottlenecks and AI incompetency under indeterministic les. She is a professor at Simon Fraser University, Burnaby, conditions or with insufficient data. The IAS framework, British Columbia, V5A 1S6, Canada, and serves as editor-inICD methodologies, and IMPACTS trust model have been chief of IEEE Transactions on Human–Machine Systems. validated through a series of defense concept development She served as IEEE Division X delegate/director, IEEE Sysand evaluation activities in large-scale international militems, Man, and Cybernetics Society (SMCS) president, and tary exercises. These defense R&D successes have been recIEEE Circuits and Systems Society (CAS) president. Her ognized by military, academic, industry, and government research interests include communication networks and authorities as well as international organizations for satisfydynamical systems. She is a Fellow of IEEE and a Distining the growing demands for brain-inspired autonomous guished Lecturer of the SMCS and CAS. systems. Novel computing theories, process capabilities, Konstantinos N. Plataniotis ([email protected]. and validation mechanisms will be developed for broader ca) is a professor and the Bell Canada chair in multimedia innovations in the defense and general industries. at the University of Toronto, Toronto, Ontario, M5S 1A1, Canada. His research interests include machine learning and About the Authors signal processing and their applications in imaging systems, Ming Hou ([email protected]) earned his Ph.D. degree communications systems, and knowledge media design sysfrom the University of Toronto, Canada in 2002. He is a tems. He is a Fellow of IEEE, the Engineering Institute of senior defense scientist and the principal authority of Canada, the Canadian Academy of Engineering/L’ Academie human–technology interactions within the Department of Canadienne Du Genie, and a registered professional engiNational Defence (DND), Toronto, Ontario, M3K 2C9, Cananeer in Ontario. He is the general cochair of the 2027 da. He is responsible for delivering technological solutions, IEEE International Conference on Acoustics, Speech, and science-based advice, and evidence-based policy recomSignal Processing. mendations to senior DND decision makers and their Sam Kwong ([email protected]) earned his B.Sc. national and international partners. He is an advisor and integrator of the Canadian government CAD$1.6 billion degree from the State University of New York at Buffalo, his IDEaS program with responsibilities for guiding national M.A.Sc. degree in electrical engineering from the University R&D activities in artificial intelligence, robotics, and telepof Waterloo in Canada, and his Ph.D. degree from Fernuniresence. He is the cochair of the Human Factors Specialist versität Hagen, Germany. He is a professor in the DepartCommittee within the international Joint Capability Group ment of Computer Science at the City University of Hong Unmanned Aircraft Systems. Kong, Hong Kong, M8100, Hong Kong. He is an associate Yingxu Wang ([email protected]) earned his Ph.D. editor of leading IEEE transaction journals, such as IEEE degree in computer science from Nottingham Trent UniverTransactions on Evolutionary Computation, IEEE Transsity, U.K., in 1998. He is a full professor in the Department of actions on Industrial Informatics, and IEEE Transactions Electrical and Software Engineering and the Hotchkiss on Cybernetics. He has coauthored three research books, sociotechnical verification and validation system-of-systems processes. Therefore, a framework for understanding trust and ethics in the context of verification regimes must be identified or developed to guide the implementation of the MoT and EDR in the verification process for optimal social benefits and impacts. To support this endeavor, the IAS framework and associated ICD approach and IMPACTS model have provided a sophisticated architecture, systematic methodology, and structured process for developing brain-inspired autonomous systems.
18
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE Apri l 2022
eight book chapters, and more than 400 technical articles. He is a Fellow of IEEE and serves as the 2022–2023 IEEE Systems, Man, and Cybernetics Society president. MengChu Zhou ([email protected]) earned his B.S. degree from Nanjing University of Science and Technology, China, in 1983, his M.S. degree from Beijing Institute of Technology, China, in 1986, and his Ph. D. degree from Rensselaer Polytechnic Institute, Troy, New York, in 1990. He then joined New Jersey Institute of Technology, Newark, New Jersey, 07102, USA, where he is now a distinguished professor. His research interests include Petri nets, intelligent automation, the Internet of Things, and big data analytics. He has authored more than 900+ publications, including 12 books, 600+ journal articles (500+ in IEEE transactions), 29 patents, and 29 book chapters. He is a Fellow of IEEE, the International Federation of Automatic Control, American Association for the Advancement of Science, Chinese Association of Automation, and National Academy of Inventors. Edward Tunstel ([email protected]) earned his B.S. and M.E. degrees from Howard University, Washington, D.C., and his Ph.D. degree from the University of New Mexico, Albuquerque, New Mexico, in 1996. He served as a leader in robotics and autonomy with the NASA Jet Propulsion Laboratory for nearly two decades, Johns Hopkins Applied Physics Laboratory for a decade, and Raytheon Technologies Research Center for several years. He is CTO with Motiv Space Systems, Pasadena, California, 91107, USA. His research interests include deployment of robotic and autonomous systems, having authored 170 publications in those areas. He is a Fellow of IEEE and served as the 2018–2019 IEEE Systems, Man, and Cybernetics Society president. Imre J. Rudas ([email protected]) earned his Ph.D. degree in robotics and his Doctor of Science from the Hungarian Academy of Sciences in 1987 and 2004, respectively. He received his Doctor Honoris Causa degree from the Technical University of Košice, Slovakia, “Polytechnica” University of Timisoara, Romania, Óbuda University, and Slovak University of Technology in Bratislava. He is the founder and Professor Emeritus of Óbuda University, Budapest, H-1034, Hungary. He has published 22 books and 850+ articles with 7,000+ citations. His research interests include computational cybernetics, robotics, and computational intelligence. He is a Fellow of IEEE and served as the 2020–2021 IEEE Systems, Man, and Cybernetics Society president. Janusz Kacprzyk ([email protected]) is a full professor at the Systems Research Institute, Polish Academy of Sciences, Warsaw, 01-447, Poland, and a full member of the European Academy of Sciences and Arts, Spanish Royal Academy of Economic and Financial Sciences, and Bulgarian Academy of Sciences. He is the recipient of six honorary doctorates. His research interests include computational and artificial intelligence and fuzzy logic in systems, decision, and control. He has authored seven books, 150 volumes, and 650+ articles, including about 100 in Web of Sci
ence journals. He is editor-in-chief of Springer’s seven book series. He is listed in the 2020 “World’s 2% Top Scientists.” He is a Fellow of IEEE, the International Fuzzy Systems Association, European Association for Artificial Intelligence, Mexican Society for Artificial Intelligence, and the president of the Polish Operational and Systems Research Society. Henry Leung ([email protected]) is a professor of electrical and computer engineering at the University of Calgary, Calgary, Alberta, T37 3H7, Canada. He has authored more than 300 journal articles in the areas of signal and image processing, data mining, information fusion, machine learning, the Internet of Things, and sensor networks. He also holds more than 15 patents. He is the editor of the Springer book series on “Information Fusion and Data Science.” He has been an associate editor of journals such as International Journal on Information Fusion, IEEE Transactions on Aerospace and Electronic Systems, IEEE Signal Processing Letters, and IEEE Circuits and Systems Magazine. He is a Fellow of IEEE and the International Society for Optical Engineering. References [1] T. B. Sheridan, Humans and Automation: System Design and Research Issues. Santa Monica, CA, USA: Wiley-Interscience, 2002. [2] J.-F. Bonnefon, A. Shariff, and I. Rahwan, “The social dilemma of autonomous vehicles,” Science, vol. 352, no. 6293, pp. 1573–1576, 2016, doi: 10.1126/science.aaf2654. [3] A. Lacher, R. Grabowski, and S. Cook, “Autonomy, trust, and transportation,” in Proc. AAAI Spring Symp. Ser., 2014, pp. 1–8. [4] M. Harlamova and M. Kirikova, Towards the Trust Handling Framework for Industry 4.0 (Frontiers in Artificial Intelligence and Applications), IOS Press, 2018, pp. 49–64, doi: 10.3233/978-1-61499-941-6-49. [5] F. Mannhardt, S. A. Petersen, and M. F. Oliveira, “A trust and privacy framework for smart manufacturing environments,” J. Ambient Intell. Smart Environ., vol. 11, no. 3, pp. 201–219, 2019, doi: 10.3233/AIS-190521. [6] H. M. Roff and R. Moyers, “Meaningful human control, artificial intelligence and autonomous weapons,” presented at the Informal Meeting of Experts on Lethal Autonomous Weapon Systems, UN Convention on Certain Conventional Weapons, 2016. [7] K. Bekiroglu, S. Srinivasan, E. Png, R. Su, and C. Lagoa, “Recursive approximation of complex behaviours with IoT-data imperfections,” IEEE/CAA J. Automatica Sinica, vol. 7, no. 3, pp. 656–667, 2020, doi: 10.1109/JAS.2020.1003126. [8] Y. Wang et al., “On future development of autonomous systems: A report of the plenary panel at IEEE ICAS’21,” in Proc. 1st IEEE Conf. Auton. Syst., 2021, pp. 1–9, doi: 10.1109/ICAS49788.2021.9551188. [9] M. Hou, S. Banbury, and C. Burns, Intelligent Adaptive Systems: An InteractionCentered Design Perspective. Boca Raton, FL, USA: CRC Press, 2014. [10] Majority Staff of the U.S., “Final committee report: The design, development, and certification of the Boeing 737 Max,” House Committee on Transportation and Infrastructure, Washington DC, USA, 2020. [Online]. Available: https://transportation.house. gov/download/20200915-final-737-max-report-for-public-release [11] N. T. S. Board, “Loss of thrust in both engines,” US Airways Flight 1549 Airbus Industrie A320-214, N106US, 2010. [12] E. Awad et al., “The moral machine experiment,” Nature, vol. 563, no. 7729, pp. 59–64, 2018, doi: 10.1038/s41586-018-0637-6. [13] A. Girma et al., “IoT-enabled autonomous system collaboration for disaster-area management,” IEEE/CAA J. Automatica Sinica, vol. 7, no. 5, pp. 1249–1262, 2020, doi: 10.1109/JAS.2020.1003291.
Ap ri l 2022
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE
19
[14] Y. Wang et al., “Towards a theoretical framework of autonomous systems under-
[35] R. Arrabito, M. Hou, S. Banbury, B. Martin, F. Ahmad, and S. Fang, “A review of
pinned by intelligence and systems science,” IEEE/CAA J. Automatica Sinica, vol. 8,
human factors research performed from 2014 to 2017 in support of the Royal Canadian
no. 1, pp. 52–63, 2021, doi: 10.1109/JAS.2020.1003432.
Air Force remotely piloted aircraft system project,” J. Unmanned Veh. Syst., vol. 9, no.
[15] IEEE Recommended Practice for Assessing the Impact of Autonomous and Intel-
1, pp. 1–20, 2020, doi: 10.1139/juvs-2020-0012.
ligent Systems on Human Well-Being, IEEE Standard 7010-2020.
[36] M. Hou, G. Ho, R. G. Arrabito, S. Young, and S. Yin, “Effects of display mode and
[16] Y. Wang et al., “On the philosophical, cognitive and mathematical functions of
input method for handheld control of micro-aerial vehicles for a reconnaissance
symbiotic autonomous systems,” Philosoph. Trans. Roy Soc. A, Math. Phys. Eng. Sci.,
mission,” IEEE Trans. Human-Mach. Syst., vol. 43, no. 2, pp. 149–160, 2013, doi:
vol. 39, no. 2207, pp. 1–20, 2021, doi: 10.1098/rsta.2020.0362.
10.1109/TSMC.2013.2239595.
[17] D. D. Schmorrow and A. A. Kruse, “Augmented cognition,” in Berkshire Encyclo-
[37] Guidance on Sense and Avoid for Unmanned Aircraft Systems, NATO STANREC
pedia of Human-Computer Interaction, W. S. Bainbridge, Ed. Great Barrington, MA,
4811, 2018.
USA: Berkshire Publishing Group LLC, 2004, pp. 54–59.
[38] Human Systems Integration Guidance for Unmanned Aircraft Systems, NATO
[18] Y. Wang, “Cognitive informatics: A new transdisciplinary research field,” Brain
STANREC 4685, 2022.
Mind, vol. 4, no. 2, pp. 115–127, 2003, doi: 10.1023/A:1025419826662.
[39] M. Hou, “Context-based and interaction-centred design for UAS GCS, NATO
[19] P. M. Fitts et al., “Human engineering for an effective air-navigation and traffic-
lecture series AVT-274 on unmanned air vehicles: Technological challenge, concepts
control system,” National Research Council, Division of Anthropology and Psychology,
of operations, and regulatory issues,” Defence Research and Development Canada,
Committee on Aviation Psychology, Washington, DC, USA, 1951. [Online]. Available:
Ottawa, Canada, DRDC-RDDC-2022-N063, 2017–2019.
https://apps.dtic.mil/sti/citations/ADB815893
[40] M. Hou, “Training needs analysis for UAS operators, NATO lecture series AVT-274
[20] M. R. Endsley and E. O. Kiris, “The out-of-the-loop performance problem and level
on unmanned air vehicles: Technological challenge, concepts of operations, and regu-
of control in automation,” Human Factors, vol. 37, no. 2, pp. 381–394, Jun. 1995, doi:
latory issues,” Defence Research and Development Canada, Ottawa, Canada, DRDC-
10.1518/001872095779064555.
RDDC-2022-N062, 2017–2019.
[21] R. Parasuraman and D. H. Manzey, “Complacency and bias in human use of auto-
[41] M. Hou, “Human systems integration and its guidance for UAS GCS certification,
mation: An attentional integration,” Human Factors, vol. 52, no. 3, pp. 381–410, 2010,
NATO lecture series AVT-274 on unmanned air vehicles: Technological challenge,
doi: 10.1177/0018720810376055.
concepts of operations, and regulatory issues,” Defence Research and Development
[22] P. de Vries, C. Midden, and D. Bouwhuis, “The effects of errors on system trust,
Canada, Ottawa, Canada, DRDC-RDDC-2022-N064, 2017–2019.
self-confidence, and the allocation of control in route planning,” Int. J. Human-
[42] M. Hou, G. Ho, and D. Dunwoody, “IMPACTS: A trust model for human-autonomy
Comput. Stud., vol. 58, no. 6, pp. 719–735, 2003, doi: 10.1016/S1071-5819(03)00039-9.
teaming,” Human-Intell. Syst. Integration, vol. 3, no. 2, pp. 79–97, 2021, doi: 10.1007/
[23] D. A. Norman and S. W. Draper, User Centered System Design: New Perspectives on
s42454-020-00023-x.
Human-Computer Interaction. Hillsdale, NJ, USA: Lawrence Erlbaum Associates, 1986.
[43] P. Zhang, M. Zhou, and Y. Kong, “A double-blind anonymous evaluation-based trust
[24] Human-Centred Design Processes for Interactive Systems, ISO 13407:1999.
model in cloud computing environments,” IEEE Trans. Syst., Man, Cybern. Syst., vol.
[25] K. J. Vicente, “Coherence-and correspondence-driven work domains: Implica-
51, no. 3, pp. 1805–1816, 2021, doi: 10.1109/TSMC.2019.2906310.
tions for systems design,” Behav. Inf. Technol., vol. 9, no. 6, pp. 493–502, 1990, doi:
[44] P. Zhang, M. Zhou, Q. Zhao, A. Abusorrah, and O. Bamasak, “A performance-opti-
10.1080/01449299008924262.
mized consensus mechanism for consortium blockchains consisting of trust-varying
[26] C. D. Wickens, Engineering Psychology and Human Performance, 2nd ed. New
nodes,” IEEE Trans. Netw. Sci. Eng., vol. 8, no. 3, pp. 2147–2159, 2021, doi: 10.1109/
York, NY, USA: Harper Collins, 1992.
TNSE.2021.3079415.
[27] D. A. Norman, “Human-centered design considered harmful,” Interactions, vol.
[45] J. H. Cho, K. Chan, and S. Adali, “A survey on trust modeling,” ACM Comput.
12, no. 4, pp. 14–19, 2005, doi: 10.1145/1070960.1070976.
Surv., vol. 48, no. 2, pp. 1–40, 2015, doi: 10.1145/2815595.
[28] J. Rasmussen, Information Processing and Human-Machine Interaction: An
[46] T. B. Sheridan, “Individual differences in attributes of trust in automation: mea-
Approach to Cognitive Engineering. New York, NY, USA: Elsevier, 1986.
surement and application to system design,” Frontiers Psychol., vol. 10, p. 1117, May
[29] M. Hou, H. Zhu, M. Zhou, and R. Arrabito, “Optimizing operator-agent interac-
2019, doi: 10.3389/fpsyg.2019.01117.
tion in intelligent adaptive interface design: A conceptual framework,” IEEE Trans.
[47] K. E. Schaefer, J. Y. C. Chen, J. L. Szalma, and P. A. Hancock, “A meta-analysis of
Syst., Man, Cybern. C, Appl. Rev.* (1995–2012), vol. 41, no. 2, pp. 161–178, 2011, doi:
factors influencing the development of trust in automation,” Human Factors, vol. 58,
10.1109/TSMCC.2010.2052041.
no. 3, pp. 377–400, 2016, doi: 10.1177/0018720816634228.
[30] M. Hou, “Intelligent adaptive system: A strategic perspective on delivering science
[48] D. McColl, K. Heffner, S. Banbury, M. Charron, R. Arrabito, and M. Hou, “Author-
technology and innovation on effective human-AI/autonomy integration,” Defence
ity pathway: Intelligent adaptive automation development for a UAS GCS,” in Proc.
Research and Development Canada, Ottawa, Canada, DRDC-RDDC-2021-B020, 2021.
Human Comput. Interaction Conf. Eng. Psychol. Cognit. Ergonomics, Perfor-
[31] M. Hou and C. M. Fidopiastis, “A generic framework of intelligent adaptive learn-
mance, Emotion Situation Awareness, Vancouver, Canada, 2017, pp. 329–342, doi:
ing systems: From learning effectiveness to training transfer,” Theor. Issues Ergonom.
10.1007/978-3-319-58472-0_26.
Sci., vol. 18, no. 2, pp. 167–183, 2016, doi: 10.1080/1463922X.2016.1166405.
[49] J. Bartik et al., “TTCP Autonomy Strategic Challenge (ASC) allied IMPACT final
[32] M. Hou, C. Kramer, C. Banbury, M. Lepard, and K. Osgoode, “Questioning technique
report,” Defence Research and Development Canada, Ottawa, Canada, TTCP Tech.
review and scenario specification for the CF counter-IED operator training course,”
Rep. TR-ASC-01-2020, DRDC-RDDC-2021-N062, 2020.
Defence Research and Development Canada, Ottawa, Canada, Tech. Rep. 2013-061, 2013.
[50] C. Baber, “Book review: Intelligent adaptive systems: An interaction-cen-
[33] M. Hou, “QuestionIT (Intelligent Tutor for Questioning Technique),” Defence
tered design perspective,” Ergonomics, vol. 60, no. 10, pp. 1458–1459, 2017, doi:
Research and Development Canada, Ottawa, Canada, 1416-12/016CA, 2014.
10.1080/00140139.2017.1330498.
[34] M. Hou, R. D. Kobierski, and M. Brown, “Intelligent adaptive interfaces for the
[51] IEEE Model Process for Addressing Ethical Concerns During System Design,
control of multiple UAVs,” J. Cognit. Eng. Decis. Making, vol. 1, no. 3, pp. 327–362,
IEEE Standard 7000-2021.
2007, doi: 10.1518/155534307X255654.
20
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE Apri l 2022
©SHUTTERSTOCK.COM/AUDIO UND WERBUNG
I
nternet of Things (IoT) and advanced communication technologies have demonstrated great potential to manage residential energy resources by enabling demand-side management (DSM). Home energy management systems (HEMSs) can automatically control electricity production and usage inside homes
by Hafiz Majid Hussain, Arun Narayanan, Subham Sahoo, Yongheng Yang, Pedro H.J. Nardelli, and Frede Blaabjerg
using DSM techniques. These HEMSs wirelessly collect information from hardware installed in the power system and homes with the objective of intelligently and efficiently optimizing electricity usage and minimizing costs. However, HEMSs can be v ulnerable to cyberattacks that target the electricity pricing model. The
Home Energy Management Systems Operation and Resilience of Heuristics Against Cyberattacks
Digital Object Identifier 10.1109/MSMC.2021.3114139 Date of current version: 25 April 2022
2333-942X/22©2022IEEE
Ap ri l 2022
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE
21
operators to modify load energy cyberattacker manipulates the demand profiles to achieve differpricing information collected by Home energy ent objectives, such as optimizing a customer’s HEMS to misguide management systems the usage of renewable energy, its algorithms toward nonoptimal reducing peak loads, or moving solutions. The customer’s eleccan automatically some loads to off-peak times, tricity bill increases, and addicontrol electricity such as nighttime and weekends. tional peaks are created without Such DSM has become important being detected by the system production and usage and popular recently because it operator. This article introduces inside homes using facilitates the incorporation of dema nd response (DR)-ba sed DSM techniques. renewable energy sources (RESs) DSM in HEMSs and discusses DR into the power system by customopt i m i z at ion u si ng heu r i st ic ers. At the same time, grid operaalgorithms (HAs). Moreover, it tions are significantly impacted addresses the possibilities and by the active participation of customers in electricity impacts of cyberattacks, their effectiveness, and the dispatch. To implement DSM and optimize electricity degree of resilience of HAs against them. This artiusage, residential customers often employ HEMSs. cle also opens research questions and shows proThese play a significant role in the energy management spective directions. of the residential sector and allow the exchange of energy consumption information with the utility to improve Home Energy Management Systems the energy profile and reliability of the power grid. Smart grid technologies and smart meters have enabled An HEMS (Figure 1) is an information and managecustomers to know their demand profiles in greater ment system to automatically (or semiautomatically) deta il while helping electr icity gr id operators to monitor and control the electrical energy production improve the efficiency and reliability of the power sysa nd usage within a household by processing the tem [1]. This encourages both customers and grid
Power Generation Sources
Smart Home 1
Smart Home N
SMj Home Appliances
Home Appliances Utility Industry
SMN
SM1
HEMS
HEMS
Power Flow Communication Flow
DR Data Management Pricing Programs System Market Advanced Metering Infrastructure
Figure 1. An HEMS in the electrical power system. SM: smart meter (subscript indicates j = {1,2,3..N}). 22
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE Apri l 2022
we brief ly introduce the main information collected from hardideas behind DR and the schedware installed in the electrical The continuous uling algorithms. power system a nd household. integration of RESs The typical objective of an HEMS Demand Response is to minimize the customer’s into the power system The goal of an HEMS is to enable costs. Bidirectional communicahas made it important and support DSM to meet specific tion a mong the HEMS, sma r t objectives, such as the minimizameters, utility, and power grid to enable effective tion of customers’ electricity bills, enables the HEMS to meet its DSM to match the utility costs, or system costs. DSM objectives by, for example, implepower supply with is typically achieved by offering menting a peak-shaving strategy financial incentives to customers, while considering the electricity the load. i nduci ng behav iora l cha nges price signal. through education, using higherAn IoT network, along with an efficiency loads, increasing diveradvanced metering infrastrucsity factors, using distributed energy resources, or other ture (AMI), supports the bidirectional communication measures [3]. The continuous integration of RESs into the and enables robust data management systems, strong power system has made it important to enable effective network connectivity, and smart metering systems. DSM to match the power supply with the load. The deployment of an AMI makes it possible for smart DR methods, which offer financial incentives to cusmeters to measure and collect useful information, such tomers, are popular and highly researched techniques to as the energy consumption, available (generated) enerachieve DSM since they incentivize RES integration along gy, or energy price in the next hour, in a precise and with DSM. DR is defined as [4]: t i me l y m a n n e r. Mo r e o v e r, t h i s i n fo r m a t io n i s a tariff or program established to motivate changes in exchanged between the HEMS and utility simultaneelectric usage by end-users from their normal conously in real time. As a result, customers can take sumption patterns in response to changes in the price pa r t i n DSM st r at eg ie s a nd m a n a ge t he ener g y of electricity over time, or to give incentive payments demand effectively. designed to induce lower electricity use at times of Figure 2 illustrates the operations of a typical HEMS. high wholesale market prices or when system reliabilFour components—the data aggregator (DA), software ity is jeopardized. and network management (SNM), appliance manageHEMSs nearly always employ DR methods to achieve ment system (AMS), and HA—interface with each other their goals. to form the HEMS. The DA receives energy pricing and production information and sends this to the SNM and HA. The AMS component collects data about appliances, such as energy consumption, operation time interval, Data Pricing Information Aggregation data received from user interfaces, and so on, and Energy Transaction exchanges them with the HA and SNM. Thereafter, the HA executes the scheduling task and sends the results (a Software and new schedule and so on) to the SNM, which operates as Network the primary control and management component, manManagement aging the accumulated data of the DA, HA, and AMS components and processing the flow of instructions in Heuristic the network. Algorithms HEMSs and their characteristics have been extensively investigated in the last decade, and a comprehensive descr iption of HEMS a rchitectu res, DSM Appliances Appliance approaches, smart grid technologies, communication Fixed Power Management Flexible Power protocols, and various decision-making algorithms System can be found in [2]. This article focuses on the operational aspects of HEMSs and assesses their resilience User Interfaces against a specific type of cyberattack. Such an attack is def i ned by fa ke pr ice sig na ls that a re used a s inputs to the HEMSs to alter their load schedule. To the best of the authors’ knowledge, this important a spect ha s not yet been studied in the literature. Before we discuss the details of the proposed study, Figure 2. The typical operations of an HEMS.
Ap ri l 2022
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE
23
the peak time interval (which DR can be categorized into yields a higher cost) or off peak two types: incentive- and priceDR methods, which (at a lower cost as a result of less based programs [5]. In an incenoffer financial stress on the grid). tive-based program, customers In this case, DR algorithms participate by reallocating their incentives to depend on the flexibility offered energy consumption in off-peak customers, are by home appliances. An applihours, in response to which a ance is flexible if its energy conreward (a bill credit or payment) popular and sumption can be shifted in time is given to them. Incentive-based highly researched within the boundaries of end-user programs involve direct load concom for t r e q u i r ement s wh i le trol, load curtailing, emergency techniques to maintaining the total consumpDRs, and so on. achieve DSM since tion [8]. Home appliances can be On the other hand, a price-based they incentivize RES divided into two types based on program is a more indirect means their characteristics and prioriof achieving DR. In this approach, integration along ties [9], [10]: different pricing signals are sent with DSM. at varying times to customers. As ◆◆ Fixed-power appliances have a a result, customers are induced fixed power consumption proto reduce their energy consumpfile and operating time, e.g., tion at cer ta in times to take ceiling fans, lamps, and TVs. advantage of possible monetary benefits. Price-based ◆◆ Flexible-power appliances can be controlled, and programs include time-of-use (TOU) tariffs, real-time their energy consumption profiles can be scheduled pricing, inclined block rate, critical peak pricing, and by the HEMS. Their operation can be controlled by day-ahead pricing [2], [6], [7]. In recent research, priceincentive- or price-based programs. These loads can based DR has been widely studied in the residential secbe further categorized into two types—uninterrupttor, particularly in HEMSs. ible and interruptible—depending on whether their For price-based DR, the price tariff scheme, i.e., the operations can be interrupted or not. Table 1 lists price bands for different designated time intervals, the appliance classes of fixed and flexible home including off-peak, midpeak, and peak hours, is imporappliances with their power ratings and operating tant. The TOU tariff scheme is widely used in many times [9]–[11]. countries for customers in the residential sector. It provides the average electricity cost of power generaHeuristic Scheduling Algorithms tion during different time periods, thereby enabling Many techniques have been explored to exploit the customers to manage their energy usage voluntarily. flexibility in home appliances and perform DR-based Customers have flexibility to use electricity either in optimization. A typical approach is to cleverly adapt optimization techniques to solve linear and nonlinear objective functions. Recently, artificial intelligence (AI)-based methods have also become popular. HeurisTable 1. Home appliance characteristics: tic scheduling (HS) algorithms comprise an important The type, power rating (PR), and group of techniques to realize energy optimization and operating time (OT). load-shifting operations in HEMSs. Many HAs have been explored previously, depending on the problem Appliance Type PR (kWh) OT (h) setup and conditions [2], [7], [11]–[19]. Among the variCeiling fan Fixed 0.075 14 ous optimization techniques, the genetic algorithm (GA) and harmony search algorithm (HSA) are two imporLamp Fixed 0.1 13 tant ones that are particularly suitable for solving conTV Fixed 0.48 7 straint-optimization-based scheduling problems and the flexible selection criteria of achieving an optimal Oven Fixed 2.3 6 (balanced) combination of exploration and exploitation Washing Flexible (uninterruptible) 0.7 8 [11], [20], [21]. machine Iron
Flexible (uninterruptible)
1.8
7
Air conditioner
Flexible (interruptible)
1.44
10
Water heater
Flexible (interruptible)
4.45
8
24
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE Apri l 2022
Genetic Algorithm GA is a widely applied algorithm due to its fast computational time and easy implementation of many complex problems [22]. It is a metaheuristic algorithm i n spi red by t he t heor y of nat u ra l evolut ion a nd
Representative Simulations for Demand Response in Home Energy Management Systems Some simulation results are now provided to demonstrate the performance of the optimization algorithms GA and HSA. As the household loads, the eight appliances listed in Table 1 are investigated with the given power ratings and operating time periods. Since the uninterruptible appliances cannot be shifted after they start
operating, the HEMS schedules the operation of the iron after the washing machine. Interruptible appliances, on the other hand, are scheduled based on the pricing signal in any time period. The energy consumption of the household appliances for one day (starting from 12 a.m. to 12 a.m. the following day) with a scheduling resolution of 1 h (t) is considered. The TOU pricing tariffs for the summer (1 May to 31 October 2019) and winter (1 November 2018 to 30 April 2019) seasons are taken from [24].
Without HEMS With GA-HEMS
20
0 12 a.m.
4 a.m.
8 a.m.
12 p.m.
4 p.m.
8 p.m. 12 a.m.
Electricity Cost (Cents)
Time (h) 100 80
Without HEMS With HSA-HEMS
60 40 20 0 12 a.m.
4 a.m.
8 a.m.
12 p.m.
4 p.m.
8 p.m.
12 a.m.
8 p.m.
12 a.m.
8 p.m.
12 a.m.
Time (h)
Electricity Cost (Cents)
(a) 200 Without HEMS With GA-HEMS
150 100 50 0 12 a.m.
4 a.m.
8 a.m.
12 p.m.
4 p.m.
Time (h) Electricity Cost (Cents)
Harmony Search Algorithm HSA is a popular metaheuristic algorithm inspired by the musical improvisation process [23]. Consider a music orchestra that improvises to find and perform the most harmonious and melodious music. Each musician in an orchestra corresponds to a decision variable, and an instrument’s pitch range corresponds to the set of possible values of the decision variable. The musical harmony produced by the musicians at a certain time can be considered as the solution vector for an iteration. An audience’s aesthetic judgment of the music can be related to the fitness of the objective function. Just like a musical orchestra attempts to find (or play) the best music possible by improving it over time, the optimization algorithm aims to progressively find the optimal solution. Thus, HSA is an idealized mapping from qualitative improvisation into a quantitative formulation, where musical harmony concepts are applied to an optimization process.
Electricity Cost (Cents)
evolutional processes like genetic inheritance and natural selection. GA is an iterative process in which a population of potential candidate solutions is first randomly generated. The population in each iteration is called a generation. All of the individual candidates (known as genes) in the population are then evaluated using a fitness function (i.e., the problem objective). The best candidates are stochastically selected from the current generation, and their genomes are modified by r e com bi n a t ion (c r o s s over) a nd replacement (mutation) to form a new generation of candidate solutions, which 100 is then used in the next iteration. The 80 stopping criteria for the algorithm are the maximum population size and best 60 candidate allocation that satisfy the 40 objective function.
200
Without HEMS With HSA-HEMS
150 100 50 0 12 a.m.
4 a.m.
8 a.m.
12 p.m.
4 p.m.
Time (h) (b)
Figure 3. The electricity costs per hour under the TOU pricing scheme
for a day each in the (a) summer and (b) winter seasons without and with the two Has: GA and HSA.
Ap ri l 2022
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE
25
electricity market or even cause Figure 3 presents a comparison severe failures. They delineated of the electricity costs for three GA is a metaheuristic defensive measures against two cases—without an HEMS as well algorithm inspired classes of data integrity attacks: a s w it h a GA- a nd a n HSAsc a l i ng (t he met er rea d s a n HEMS—in the summer and winby the theory of amplified version of the actual ter seasons. The deployment of an natural evolution prices) and delaying (the meter HSA- and a GA-HEMS led to lower uses old pr ices). I n [29], the total costs compared to the case and evolutional authors systemically examined without an HEMS. The energy processes like genetic the arbitrary injection of pricing cost wa s h ig hest w ithout a n (data) signals and proposed counHEMS because most of the energy inheritance and termeasures based on a cumulawas used in either the peak or natural selection. tive sum control chart technique midpeak times. Between the two to identify the attacks. The injeca lgor it h m s, t he HSA-HEMS tion of false data creates a disparreduced the cost by 43.55% and ity between the generated and 11.91% in the summer and winter consumed power, which subsequently leads to two major seasons, respectively, while the GA-HEMS reduced it by problems: 1) the instability of the entire system and 2) an 23.37% and 18.91%, respectively. increase in the operational costs by the addition of forged data to the electricity market [30]–[32]. Cyberattacks Figure 4 depicts possible cyberattacks on a cyberIn a smart grid, the real-time exchange of information, physical system comprising the communication infraespecially data collected from smart meters, electricity structure of various components associated with a pricing markets, and utility companies, requires a secure smart grid connected to an end-user household. The and protective layer of the communication channel [25]. utility collects infor mation related to the energy However, the complex structure of the smart grid and prodemand (generation, consumption, and price) through liferation of smart devices make it vulnerable to cyberthe AMI and transmits this information to the smart attacks. A typical cyberattack in a smart grid is the meters and end users through an IoT or Wi-Fi network. injection of false data into the system to distort the enerThe hierarchical communication infrastructure gy demand, grid network states, and electricity pricing shown is exposed to three kinds of cyberattacks [32]. signals [26], [27]. First, an adversary can attack the utility’s main system (computer devices) and change the pricing curve. SubHow Cyberattacks Work sequently, this information is sent to the end users, and, Tan et al. [28] studied the impact of security threats on a based on the fake price, the HEMS schedules their real-time pricing system, which could destabilize the loads. Second, an attacker can directly attack smart meters at or near the enduser household and tamper with the received (or t ra n sm it ted) dat a . A n Smart Meter adversary can also attack any access Utility point in the Wi-Fi network, create a (fake) access point, and send false pricing data to the smart meter.
Attacker 1
Attacker 2 Power Grid
End User Power and Communication Direct Attack False Information Attack
Figure 4. A cyberattack on a cyberphysical system: Attacker 1
attempts to inject the wrong pricing data or alter the energy demand information, and Attacker 2 attempts a direct attack on the utility to change the energy demand and/or production. 26
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE Apri l 2022
Cyberattack Scenario Consider an HEMS that employs HS algorithms to perform DR-based optimization based on the received price signals. As discussed in [29], a smart meter or other receiver can often be hacked with minimum effort due to the lack of security measures. Let us examine a scenario where an attacker has the resources to hack into a smart meter and inject corrupted (price) information. The cyberattacker aims to mislead the heuristics to induce a higher electricity bill or peak
Resilience Index (%)
Electricity Cost (Cents)
Electricity Cost (Cents)
demand by modifying the peak prices arbitrarily, which increasHSA is an idealized es the mismatch between the gene r a t e d e ne r g y a nd e ne r g y mapping from demand. For example, in the case qualitative of TOU tariffs in winter, the peak time prices of 20.8 cents/kWh improvisation into occur from 7 a.m. to 11 a.m. and a quantitative from 6 p.m. to 8 a.m. [23]. The attacker can now alter these peak formulation, where prices by either shifting them to musical harmony the off-pea k ti me or si mply concepts are applied to directly lowering the prices, wh ic h , i n t u r n , i nc r e a s e s / an optimization process. decreases the electricity bill. In such a scenario, how do the designed models using GA and HSA react when the system is attacked and forged pricing information is injected? To analyze this, assume 200 that the adversary particularly targets the 150 peak prices of the energy demand, i.e., from 7 a.m. to 11 a.m. and from 6 a.m. to 100 8 a.m. Figure 5(a) presents the electricity costs for a day in winter after a cyberat50 tack has occurred. The GA-HEMS and 0 HSA-HEMS attempt to schedule the ener12 a.m. 4 a.m. 8 a.m. gy consumption as before, but the electricity costs naturally increase. However, 200 this increase is not very high: with the GA, the cost rises by 0.15% compared to 150 the optimal cost achieved earlier without 100 the cyberattack, and, with the HSA, the cost grows by 1.8%. 50 The resilience of any algorithm against cyberattacks can be characterized by mea0 12 a.m. 4 a.m. 8 a.m. suring how much the forged pricing data affect the performance of the considered system metrics (here, electricity costs) in 300 the designed scenario. A simple way to measure the resilience is by using a resil200 ience index (RI) as follows: C A - CO n # 100 (1) RI = 100 - d CO
where C A and C O represent the total electr icity cost when the system is under attack and otherwise, respectively. In both cases, the total cost is optimized by using the HEMS. Thus, the RI gives a measure of accuracy of the HA against cyberattacks. RI ! [- 3 100%] . RI = 100% means that the algorithm is extremely resilient (C A = C O) . As the amount of deviation from the optimal
cost increases, the RI decreases from the maximum of 100%, and it becomes negative when C A 2 2C O . A negative RI means that the algorithm’s performance is poor; the new cost is more than twice the actual cost. Figure 5(b) presents the RI for the designed model for a day. The GAHEMS maintains a good and somewhat constant RI across the day, whereas the HSA sometimes has a poor RI. Further, the overall RI values for GA and HSA for the entire day were 99.8% and 97.8%, respectively. Thus, even though the cyberattacker
Without HEMS With GA-HEMS
12 p.m.
4 p.m.
8 p.m.
12 a.m.
Time (h) Without HEMS With HSA-HEMS
12 p.m.
4 p.m.
8 p.m.
12 a.m.
8 p.m.
12 a.m.
Time (h) (a) With GA-HEMS With HSA-HEMS
100 0 –100 –200 12 a.m.
4 a.m.
8 a.m.
12 p.m.
4 p.m.
Time (h) (b) Figure 5. The impact on the electricity costs for a day when a
cyberattacker changes the TOU pricing. Two heuristic optimization algorithms, GA and HSA, optimize the costs for the HEMS. (a) The electricity costs per hour under the cyberattack case. (b) The resilience of the GA-HEMS and HAS-HEMS—represented by an RI.
Ap ri l 2022
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE
27
attempts to mislead the designed heuristic approaches with fake price information, both of the designed algorithms perform robustly against these attacks, providing a similar performance to the case without active management.
How MPSs will deal with such cyberattacks in the future will be critical to ensure their stability and performance.
Conclusion: Cyberattacks and Future Power Systems Modern power systems (MPSs) have added flexibility and coordination by utilizing information and communication technologies and AMI. MPSs have now gradually transitioned into a complex cyberphysical energy system (CPES). The cyber layer has made it possible for MPSs to not only become more responsive to faults and other systemic problems but also coordinate production and load energy by reacting faster and smarter to changes. Moreover, individual households are empowered to install HEMSs to manage their own production and load as well as interactions with the power system. The efficient transformation of an MPS into a CPES is doubly important today because global climate change issues have made it necessary to integrate large amounts of RESs into the power system. However, this transformation comes with a price: vulnerability to cyberattacks. MPS control and operations are more visible to external actors, and the strong interactions between the cyberphysical layers in a CPES increase the MPS’s vulnerability to cyberattacks. Moreover, power electronic converters, which are key enablers for integrating RESs into MPSs, are typically controlled by employing a hierarchical three-stage structure, namely, primary, secondary, and tertiary layers. This means that the MPSs have additional vulnerabilities and possible attack points in different layers of the system. A cyberattacker can take advantage of any software flaws or failures in any layer of the CPES and create harmful disturbances in the system. How MPSs will deal with such cyberattacks in the future will be critical to ensure their stability and performance. Advanced and resilient technologies and mitigation measures have to be developed and implemented at every level. Hierarchical stages in MPSs enforce different timescales of operation, giving great f lexibility to design mitigation techniques against cyberattacks. At the same time, these measures can also be cheated if the attacker has access to multiple points to design coordinated attacks [33]. Data-driven techniques are a computationally viable platform to identify such anomalies. Robust and resilient control strategies using watermarking [34] and state observers [35] could be smartly employed to infiltrate such cyberattacks in the primary and secondary control layer by guaranteeing faster action. 28
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE Apri l 2022
For researchers and industry practitioners, the development of countermeasures to mitigate the impacts of cyberattacks, including financial and data losses, privacy invasions, and so on, is a fascinating and highly relevant area of investigation today. After all, a safe and secure electrical power system is an important part of a safe and secure society.
Acknowledgments This article is partly supported by the Academy of Finland via EnergyNet Research Fellowship 321265/328869, and Framework for the Identification of Rare Events via Machine Learning and IoT Networks (326270; CHIST-ERA-17-BDSI-003). This research is also supported by the joint Baltic-Nordic Energy Research programme project “Guidelines for Next Generation Buildings as Future Scalable Virtual Management of MicroGrids [Next-uGrid]” (117766). About the Authors Hafiz Majid Hussain ([email protected]) received his B.S. degree in electrical engineering from the National University of Computer and Emerging Sciences, Pakistan, in 2014 and M.S. degree in electrical engineering from the University of Engineering and Technology Taxila, Pakistan, in 2017. Currently, he is pursuing his Ph.D. degree in electrical engineering from the Cyberphysical Systems Group, Lappeenranta University of Technology, Lappeenranta, 53850, Finland. He is part of the project called Building the Energy Internet as a Large-Scale Internet of Things-Based Cyberp hysical System. His research interests include demand response applications, energy resource optimization in smart grids, and information security technologies. For more information, see https:// sites.google.com/view/hafizmajidhussain/biography. He is a Member of IEEE. Arun Narayanan ([email protected]) received his B.E. degree in electrical engineering from Visvesvaraya National Institute of Technology, Nagpur, India, in 2002 and M.Sc. degree in energy technology from Lappeenranta University of Technology (LUT), Finland, in 2013. He received his Ph.D. degree from the School of Energy Systems, LUT. He is currently a postdoctoral fellow with the Cyber-Physical Systems Group, LUT, Lappeenranta, 53850, Finland. His research interests include renewable energy-based smart microgrids, electricity markets, demand-side management, energy management systems, and information and communications technology. He focuses on applying optimization, computational concepts, and artificial intelligence techniques to renewable electrical energy problems. He is a Member of IEEE.
Subham Sahoo ([email protected]) received his B.Tech. degree in electrical and electronics engineering from Veer Surendra Sai University of Technology, Burla, India, in 2014 and Ph.D. degree in electrical engineering from the Indian Institute of Technology, Delhi, New Delhi, India in 2018. He worked as a visiting student with the Department of Electrical and Electronics Engineering, Cardiff University, U.K., in 2017. Prior to the completion of his Ph.D. degree, he worked as a postdoctoral researcher in the Department of Electrical and Computer Engineering at the National University of Singapore in 2018–2019 and at Aalborg University (AAU), Denmark, in 2019–2020. He is currently an assistant professor in the Department of Energy, AAU, Aalborg, 9220, Denmark. His research interests include the control, optimization, and stability of power electronic-dominated grids; renewable energy integration; and physics-informed artificial intelligence tools for cyber physical power electronic systems. He was a recipient of the Indian National Academy of Engineering Innovative Students Project Award for his Ph.D. thesis across all of the institutes in India for 2019. He was also a distinguished reviewer for IEEE Transactions on Smart Grid in 2020. He currently serves as secretary for the IEEE Young Professionals Affinity Group, Denmark, and Joint IEEE Industry Applications Society/IEEE Industrial Electronics Society/IEEE Power Electronics Society in the Denmark Section. He is a Member of IEEE. Yongheng Yang ([email protected]) received his B.Eng. degree in electrical engineering and automation from Northwestern Polytechnical University, China, in 2009 and Ph.D. degree in energy technology (power electronics and drives) from Aalborg University, Denmark, in 2014. He was a postgraduate student with Southeast University, China, from 2009 to 2011. In 2013, he spent three months as a visiting scholar at Texas A&M University, United States. Since 2014, he has been with the Department of Energy, Aalborg University, Aalborg, 9220, Denmark, where he became a tenured associate professor in 2018. In January 2021, he joined Zhejiang University, China, where he is currently a ZJU100 professor with the Institute of Power Electronics. His current research interests include the grid integration of photovoltaic systems and control of power converters, in particular, the mechanism and control of grid-forming power converters and systems. He was the chair of the IEEE Denmark Section (2019–2020). He is an associate editor for several IEEE transactions/journals. He is the deputy editor for solar photovoltaic systems of the Institution of Engineering and Technology’s (IET’s) Renewable Power Generation. He was the recipient of the 2018 IET Renewable Power Generation Premium Award and was an outstanding reviewer for IEEE Transactions on Power Electronics in 2018. He received the 2021 Richard M. Bass Outstanding Young Power Electronics Engineer Award from the IEEE Power Electronics Society (PELS). In addition, he has received two IEEE Best Paper Awards. He is currently the secretary of the IEEE PELS Technical Com
mittee on Sustainable Energy Systems. He is a Senior Member of IEEE. Pedro H.J. Nardelli ([email protected]) received his B.S. and M.Sc. degrees in electrical engineering from the State University of Campinas, Brazil, in 2006 and 2008, respectively. In 2013, he received his doctoral degree from the University of Oulu, Finland, and State University of Campinas following a dual degree agreement. He is currently an associate professor (tenure track) in the Internet of Things (IoT) in energy systems at Lappeenranta University of Technology (LUT), Lappeenranta, 53850, Finland, and he holds the position of Academy of Finland research fellow with a project called Building the Energy Internet as a Large-Scale Internet of Things-Based Cyberphysical System, which manages the energy inventory of distribution grids as discretized packets via machine-type communications (EnergyNet). He leads the Cyberphysical Systems Group at LUT and is the project coordinator of the CHISTERA European Consortium Framework for the Identification of Rare Events via Machine Learning and IoT Networks. He is also docent at the University of Oulu on the topic of “communications strategies and information processing in energy systems.” His research interests include wireless communications, particularly as applied in industrial automation and energy systems. He received a Best Paper Award of the IEEE Power & Energy Society Innovative Smart Grid Technologies Latin America 2019 in the track “Big Data and the IoT.” For more information, visit https://sites.google. com/view/nardelli/. He is a Senior Member of IEEE. Frede Blaabjerg ([email protected]) received his Ph.D. degree in electrical engineering from Aalborg University in 1995. From 1987 to 1988, he was with ABB-Scandia, Randers, Denmark. He became an assistant professor in 1992, an associate professor in 1996, and a full professor of power electronics and drives in 1998 in the Department of Energy, Aalborg University, Aalborg, 9220, Denmark. In 2017, he became a Villum Investigator. He is honoris causa at University Politehnica Timisoara, Romania, and Tallinn Technical University in Estonia. He has published more than 600 journal papers in the fields of power electronics and its applications. He is the coauthor of four monographs and editor of 10 books in power electronics and its applications. His research interests include power electronics and its applications, such as in wind turbines, photovoltaic systems, reliability, harmonics, and adjustable-speed drives. He has received 32 IEEE Prize Paper Awards as well as the IEEE Power Electronics Society (PELS) Distinguished Service Award in 2009, Electronics-Power Electronics and Motion Control Council Award in 2010, IEEE William E. Newell Power Electronics Award in 2014, Villum Kann Rasmussen Research Award in 2014, Global Energy Prize in 2019, and 2020 IEEE Edison Medal. He was the editor-in-chief of IEEE Transactions on Power Electronics from 2006 to 2012. He was a Distinguished Lecturer for PELS from 2005 to 2007 and for the IEEE Industry Applications Society from 2010 to 2011 and 2017 to 2018. In 2019–2020, he served as president of Ap ri l 2022
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE
29
PELS. He is vice president of the Danish Academy of Technical Sciences too. He was nominated in 2014–2019 by Thomson Reuters to be among the 250 most-cited researchers in engineering in the world. He is a Fellow of IEEE.
new binary backtracking search algorithm,” Energy Buildings, vol. 138, pp. 215–227, 2017, doi: 10.1016/j.enbuild.2016.12.052. [18] K. Ma, T. Yao, J. Yang, and X. Guan, “Residential power scheduling for demand response in smart grid,” Int. J. Electr. Power Energy Syst., vol. 78, pp. 320–325, Jun. 2016, doi: 10.1016/j.ijepes.2015.11.099.
References
[19] P. H. Nardelli and F. Kühnlenz, “Why smart appliances may result in a stupid grid:
[1] R. Lu, S. H. Hong, and M. Yu, “Demand response for home energy management
Examining the layers of the sociotechnical systems,” IEEE Syst., Man, Cybern. Mag.,
using reinforcement learning and artificial neural network,” IEEE Trans. Smart Grid,
vol. 4, no. 4, pp. 21–27, 2018, doi: 10.1109/MSMC.2018.2811709.
vol. 10, no. 6, pp. 6629–6639, 2019, doi: 10.1109/TSG.2019.2909266.
[20] X. Z. Gao, V. Govindasamy, H. Xu, X. Wang, and K. Zenger, “Harmony search
[2] H. Shareef, M. S. Ahmed, A. Mohamed, and E. A. Hassan, “Review on home energy
method: Theory and applications,” Comput. Intell. Neurosci., vol. 2015, 2015, doi:
management system considering demand responses, smart technologies, and intel-
10.1155/2015/258491.
ligent controllers,” IEEE Access, vol. 6, pp. 24,498–24,509, Apr. 2018, doi: 10.1109/
[21] T. El-Ghazali, Metaheuristics: From Design to Implementation. Chichester, U.K.:
ACCESS.2018.2831917.
Wiley, 2009, vol. 9, pp. 10–11.
[3] W. Chiu, H. Sun, and H. V. Poor, “Energy imbalance management using a robust
[22] J. H. Holland et al., Adaptation in Natural and Artificial Systems: An Introduc-
pricing scheme,” IEEE Trans. Smart Grid, vol. 4, no. 2, pp. 896–904, 2013, doi:
tory Analysis with Applications to Biology, Control, and Artificial Intelligence. Cam-
10.1109/TSG.2012.2216554.
bridge, MA, USA: MIT Press, 1992.
[4] “Benefits of demand response in electricity markets and recommendations for
[23] O. Abdel-Raouf and M. A.-B. Metwally, “A survey of harmony search algorithm,”
achieving them,” U.S. Dept. Energy, Washington, DC, USA, Tech. Rep, 2006.
Int. J. Comput. Appl., vol. 70, no. 28, pp. 1818–1831, 2013, doi: 10.5120/12255-8261.
[5] P. Siano, “Demand response and smart grids—A survey,” Renew. Sustain. Energy
[24] “Time-of-Use (TOU) pricing and schedules,” powerstream.ca, 2019. https://www.
Rev., vol. 30, pp. 461–478, Feb. 2014, doi: 10.1016/j.rser.2013.10.022.
powerstream.ca/customers/rates-support-programs/time-of-use-pricing.html
[6] J. S. Vardakas, N. Zorba, and C. V. Verikoukis, “A survey on demand response pro-
[25] A. Hahn, A. Ashok, S. Sridhar, and M. Govindarasu, “Cyber-physical security tes-
grams in smart grids: Pricing methods and optimization algorithms,” IEEE Commun.
tbeds: Architecture, application, and evaluation for smart grid,” IEEE Trans. Smart
Surveys Tuts., vol. 17, no. 1, pp. 152–178, 2014, doi: 10.1109/COMST.2014.2341586.
Grid, vol. 4, no. 2, pp. 847–855, 2013, doi: 10.1109/TSG.2012.2226919.
[7] W. Fan, N. Liu, and J. Zhang, “Multi-objective optimization model for energy manage-
[26] A. Humayed, J. Lin, F. Li, and B. Luo, “Cyber-physical systems security—A
ment of household micro-grids participating in demand response,” in Proc. IEEE Innov.
survey,” IEEE Internet Things J., vol. 4, no. 6, pp. 1802–1831, 2017, doi: 10.1109/
Smart Grid Technol.-Asia (ISGT ASIA), 2015, pp. 1–6, doi: 10.1109/ISGT-Asia.2015.7387040.
JIOT.2017.2703172.
[8] P. MacDougall, B. Roossien, C. Warmer, and K. Kok, “Quantifying flexibility for
[27] F. Aloul, A. Al-Ali, R. Al-Dalky, M. Al-Mardini, and W. El-Hajj, “Smart grid security:
smart grid services,” in Proc. Power Energy Soc. Gen. Meeting (PES), Vancouver,
Threats, vulnerabilities and solutions,” Int. J. Smart Grid Clean Energy, vol. 1, no. 1,
Canada, Jul. 2013, pp. 1–5.
pp. 1–6, 2012, doi: 10.12720/sgce.1.1.1-6.
[9] H. Hussain, N. Javaid, S. Iqbal, Q. Hasan, K. Aurangzeb, and M. Alhussein, “An efficient
[28] R. Tan, V. Badrinath Krishna, D. K. Yau, and Z. Kalbarczyk, “Impact of integ-
demand side management system with a new optimized home energy management con-
rity attacks on real-time pricing in smart grids,” in Proc. ACM SIGSAC Conf. Comput.
troller in smart grid,” Energies, vol. 11, no. 1, p. 190, 2018, doi: 10.3390/en11010190.
Commun. Security, 2013, pp. 439–450, doi: 10.1145/2508859.2516705.
[10] Y. Liu, C. Yuen, R. Yu, Y. Zhang, and S. Xie, “Queuing-based energy consumption
[29] J. Giraldo, A. Cárdenas, and N. Quijano, “Integrity attacks on real-time pricing in
management for heterogeneous residential demands in smart grid,” IEEE Trans.
smart grids: Impact and countermeasures,” IEEE Trans. Smart Grid, vol. 8, no. 5, pp.
Smart Grid, vol. 7, no. 3, pp. 1650–1659, 2016, doi: 10.1109/TSG.2015.2432571.
2249–2257, 2016, doi: 10.1109/TSG.2016.2521339.
[11] Q. Zhu, J. Zhang, Y. Hou, and Y. Qiao, “The energy-saving scheduling of campus
[30] D. B. Rawat and C. Bajracharya, “Detection of false data injection attacks in
classrooms: A simulation model,” IEEE Syst., Man, Cybern. Mag., vol. 7, no. 2, pp.
smart grid communication systems,” IEEE Signal Process. Lett., vol. 22, no. 10, pp.
22–34, 2021, doi: 10.1109/MSMC.2020.3046236.
1652–1656, 2015, doi: 10.1109/LSP.2015.2421935.
[12] B. Gunay and W. Shen, “Connected and distributed sensing in buildings: Improv-
[31] L. Xie, Y. Mo, and B. Sinopoli, “Integrity data attacks in power market opera-
ing operation and maintenance,” IEEE Syst., Man, Cybern. Mag., vol. 3, no. 4, pp.
tions,” IEEE Trans. Smart Grid, vol. 2, no. 4, pp. 659–666, 2011, doi: 10.1109/
27–34, 2017, doi: 10.1109/MSMC.2017.2702386.
TSG.2011.2161892.
[13] H. M. Hussain and P. H. Nardelli, “A heuristic-based home energy management
[32] Y. Liu, S. Hu, and T.-Y. Ho, “Leveraging strategic detection techniques for smart
system for demand response,” in Proc. IEEE Conf. Ind. Cyberphys. Syst. (ICPS), 2020,
home pricing cyberattacks,” IEEE Trans. Dependable Secure Comput., vol. 13, no. 2,
vol. 1, pp. 285–290, doi: 10.1109/ICPS48405.2020.9274742.
pp. 220–235, 2015, doi: 10.1109/TDSC.2015.2427841.
[14] P. Nardelli, H. M. Hussain, A. Narayanan, and Y. Yang, “Virtual microgrid manage-
[33] S. Sahoo, T. Dragicˇevic´, and F. Blaabjerg, “Cyber security in control of grid-
ment via software-defined energy network for electricity sharing: Benefits and chal-
tied power electronic converters–challenges and vulnerabilities,” IEEE J. Emerg.
lenges,” IEEE Syst., Man, Cybern. Mag., vol. 7, no. 3, pp. 10–19, 2021, doi: 10.1109/
Sel. Topics Power Electron., vol. 9, no. 5, pp. 5326–5340, 2019, doi: 10.1109/
MSMC.2021.3062018.
JESTPE.2019.2953480.
[15] Z. Zhao, W. C. Lee, Y. Shin, and K.-B. Song, “An optimal power scheduling method
[34] S. Weerakkody, Y. Mo, and B. Sinopoli, “Detecting integrity attacks on control sys-
for demand response in home energy management system,” IEEE Trans. Smart Grid,
tems using robust physical watermarking,” in Proc. 53rd IEEE Conf. Decision Control,
vol. 4, no. 3, pp. 1391–1400, 2013, doi: 10.1109/TSG.2013.2251018.
2014, pp. 3757–3764, doi: 10.1109/CDC.2014.7039974.
[16] C. Bharathi, D. Rekha, and V. Vijayakumar, “Genetic algorithm-based demand side
[35] F. Pasqualetti, F. Dorfler, and F. Bullo, “Control-theoretic methods for cyber-
management for smart grid,” Wireless Personal Commun., vol. 93, no. 2, pp. 481–502,
physical security: Geometric principles for optimal cross-layer resilient control
2017, doi: 10.1007/s11277-017-3959-z.
systems,” IEEE Control Syst. Mag., vol. 35, no. 1, pp. 110–127, 2015, doi: 10.1109/
[17] M. S. Ahmed, A. Mohamed, T. Khatib, H. Shareef, R. Z. Homod, and J. Abd Ali,
MCS.2014.2364725.
“Real time optimal schedule controller for home energy management system using
30
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE Apri l 2022
©SHUTTERSTOCK.COM/P AND P STUDIO
Tensor-Based Knowledge Fusion and Reasoning for Cyberphysical-Social Systems Theory and Framework by Jing Yang, Laurence T. Yang, Yuan Gao, Huazhong Liu, Hao Wang, and Xia Xie
C
yberphysical-social systems (CPSS) integrate human, machine, and information into large-scale automated systems and generate complex heterogeneous big data from multiple sources. Knowledge graphs play a pivotal role in energizing the data with huge volume and uneven quality to drive CPSS intelligent applications and services,
Digital Object Identifier 10.1109/MSMC.2021.3114538 Date of current version: 25 April 2022
2333-942X/22©2022IEEE
Ap ri l 2022
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE
31
have led to the geometric growth thus attracting intense research of digital information and further i nterests from schola r s. T he The performance facilitated CPSS big data formaResource Description Framework of all subsequent tion [3], [4]. The CPSS big data (RDF) describes knowledge in the exhibits characteristics of being form of subject-predicate-object knowledge enormous in quantity, uneven in triples and interpreted as directed technology quality, updated in real time, and labeled graphs. However, the graph manifold in data sources; thus, it structure doesn’t have f lexible processes depend is difficult to intuitively manage operability and direct computabiliheavily on whether a nd utilize these data, which ty in the theoretical framework, attracts many scholars to devote although it can be understood intuthe knowledge themselves to the effective proitively. Therefore, we proposed a representation is cessing of data [5], [8]. They comtensor-based knowledge analysis expressive and mit to discovering and extracting framework in this article, which semantic information and valusuppor t s t he represent at ion, effective. able knowledge from data of diffusion, and reasoning of knowlferent sou rce s a nd d i f ferent edge graphs. First, we employ structures and storing it in the Boolean tensors to represent hetknowledge graph. Therefore, more standardized and erogeneous knowledge graphs completely. Then, we preshigh-quality data lives in the knowledge graph, which ent a ser ies of graph tensor operations for the builds the bridge from massive data that is generated by modification, extraction, and aggregation of high-order the interaction and communication between various knowledge graphs. Furthermore, we perform tensor objects to smart applications and services [9], [10]. 1-mode product operation between the knowledge graph The concept of knowledge graph became widely popular representation tensor and the entity representation tensor when it was delivered by Google’s search engine [6] in 2012. to obtain the relation path tensor, so as to infer the relaOver recent years, many large-scale knowledge bases have tionship between any two entities. Finally, we demonstrate emerged, such as YAGO, DBpedia, FreeBase, and so on, the practicality and effectiveness of the proposed model by which are built to provide users with intelligent services in implementing a case study. CPSS including semantic search, assisted intelligent question and answer, explainable artificial intelligence, and so Background on [7]. The primary task in the knowledge graph, knowledge With the popularization of mobile terminals and the representation, plays an influential role since the perfordeepening of network applications, especially the develmance of all subsequent knowledge technology processes opment of technologies such as the Internet, big data, depend heavily on whether the knowledge representation is cloud computing, and the Internet of Things (IoT), peoexpressive and effective. Web Ontology Language and RDF ple have become the most sensitive social sensors [1], schema are utilized by the World Wide Web Consortium for [2]. By further incorporating social information, the representing information on the web. They are the extencyberphysical-social systems evolve rapidly into more sions of the RDF in the data model and logical structure. complex systems that integrate human, machine, and RDF describes knowledge in the form of subject-predicateinformation, that is, CPSS. In CPSS, the seamless conobject triples, denoted by (s, p, o) for convenience. As nection of IoT, the large-scale deployment of heterogeneous sensors, and the rapid development of existing shown in Figure 1(a), the triples are visualized in a directed computing power, storage space, and network bandwidth labeled graph. It can be concluded that s, p, and o are treated in a different way, namely, the subject and object are interpreted as nodes, while Subject/Object the predicate is interpreted as edges. What’s e e e more, the graph structure doesn’t have flexPredicate 1 2 3 ible operability and direct computability in e3 (e1, r1, e3) the theoretical framework, although it can r1 XG (3, 3, 2) = 1 r1 r3 (e2, r2, e1) be understood intuitively. r2 In recent years, many scholars have care2 (e3, r3, e2) e1 r2 ried out a lot of theoretical research and r3 practical exploration in the efficient representation and calculation of multisource (a) (b) heterogeneous big data [13]. As an extension of vector and matrix, tensor, also Figure 1. The examples of the RDF data model and the tensor data known as a multiway array, is a powerful model: (a) RDF and (b) tensor. 32
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE Apri l 2022
and effective tool for large-scale multidimensional data are usually multisource and heterogeneous. CPSS big representation and analysis [12], which can provide data can be divided into three categories in structure, including unstructured data represented by videos and knowledge graphs with new perspectives and possibilities. In this article, we propose to represent RDF data as audio, semistructured data represented by XML and HTML, and structured data represented by GPS data and Boolean tensors to fully maintain the spatiotemporal electronic health records data. interweaving and complex associations of knowledge graphs without any information loss and conduct knowl2) Knowledge plane: The knowledge acquired from CPSS edge fusion and inference exploiting tensor operations. big data is more standardized and high quality. In this Specifically, subject, predicate, object in RDF triples can layer, we represent the knowledge graphs by Boolean be expressed as three orders in a Boolean tensor naturally. tensors, which can model the inherent structure of mulThe binary element in the Boolean tensor expresses the tirelational data. presence or absence of each triple. As shown in Figure 1(b), 3) Analytic plane: After representing knowledge by tenwe use a tensor with size 3 # 3 # 3 to represent the triples, sors, we then fuse the knowledge graphs extracted from different scenarios and generated continuously over and the element X G (3, 3, 2) = 1 in the tensor corresponds to time by exploiting a series of graph tensor operations. the triple (e 3, r3, e 2). Note that the tensor-based knowledge Then, we perform tensor-based knowledge reasoning on representation is adaptive and competent for the future highthe integrated knowledge graphs, which can model rich order knowledge graph since the tensors can be of any order; interactive semantic information. we just simply demonstrate how to represent the current RDF data as a third-order tensor here. What’s more, there are 4) Application plane: In terms of the different applications sufficient mathematical theories and adaptable operations such as question and answer systems, recommendation that support tensors for subsequent knowledge fusion and reasoning processes after the RDF data have been represented as tenApplication Plane sors. Different from the traditional matrix… based knowledge representation and Knowledge inference method, by further considering the Services Community Recommendation Q&A information of the data node itself and the Mining System System diversified structural relationship between different entities, the tensor-based knowlAnalytic Plane edge fusion and reasoning methods can adeTensorquately model the semantic interactions Relationship Knowledge Knowledge Based Link Graph Quality Knowledge … between entities, relations, and triples [11]. Prediction Completion Verification Reasoning Furthermore, the real-time and high-level quality requirements for intelligent applicaGraph Graph Graph Graph Tensor tions in CPSS demand exceedingly powerful Graph … Tensor Tensor Tensor Operations Subtensor knowledge processing capability, and tensorProduct Union Intersection and Fusion based knowledge reasoning approaches can also deal with the knowledge graphs that Knowledge Plane have the characteristics of incremental generation and dynamic evolution over time and I1 I1 I I1 I2 Tensor IN 2 I2 discover the potential inherent patterns and … Representation IN I3 IN dynamic evolution laws of complex behavI3 I3 iors in different scenarios [14]. Tensor-Based Framework We present a tensor-based knowledge representation, fusion, and reasoning framework for heterogeneous knowledge graphs in CPSS. As shown in Figure 2, the framework contains four planes, i.e., data plane, knowledge plane, analytic plane, and application plane. The description of the function of each layer and the relations between them are explained here: 1) Data plane: CPSS data collected directly from cyber, physical, and social spaces
Knowledge Acquire
…
Data Plane Video
Audio
XML
HTML
GPS
EHR
Unstructured Data
Semistructured Data
Data Collection
Structured Data
Figure 2. A proposed tensor-based knowledge representation, fusion,
and reasoning framework.
Ap ri l 2022
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE
33
systems, community Mining, and so on, relevant knowledge services are carried out based on the reasoning results provided by the analytic plane. Tensor-Based Knowledge Representation and Fusion The knowledge graph is often multisource and dynamic due to the high complexity and heterogeneity of the CPSS. It is necessary and meaningful to fuse the knowledge acquired from different scenarios and devices as well as that generated continuously over time. Specifically, the knowledge generated by common things in different scenarios and periods is very relevant. If the inference results depend only on any separate knowledge, it will be one-sided and inadequate. Therefore, reasoning over integrated knowledge will be beneficial to intelligent decision making. We first represent knowledge graphs as Boolean tensors, and then we propose a series of graph tensor operations for knowledge graph fusion. Graph tensor operations and definitions are summarized in Table 1. Tensor Representation We first give a general definition of knowledge graph t e n s o r representation. Given a knowledge graph G ^ A 1, A 2, f, A N , T h, A n denotes the set of entities or relations, and T denotes the facts of the structure "^a 1, a 2, f, a N h, 1 A 1 # A 2 # g # A N . The graph tensor that represents G can be defined as X G ! B I # I #g # I , where B = " 0, 1 ,, I n denote the length of set A n. X G ^a 1, a 2, f, a N h = 1 indicates the fact ^a 1, a 2, f, a N h exists; otherwise, it does not exist. 1
2
a nd so on. More for ma lized def initions a re given as follows: ◆◆ Delete edge: Given r ! A n, where A n is a relations set, the graph tensor X G - r denotes removing the relation r from G ^ A 1, A 2, f, A N , T h . X G - r can be obtained by deleting the r-th slice of tensor X G . ◆◆ Delete vertex: Given t ! A m, where A m is a entity set, the graph tensor X G -t denotes removing the entity t from G ^ A 1, A 2, f, A N , T h and the edges that are incident on t, which are denoted by R ^ t h . X G -t can be obtained by deleting the t-th and r-th, slices of tensor X G, r ! R ^ t h . ◆◆ Contract edge: Given ri, r j ! A n, where A n is a relations set, the graph tensor X r + r denotes the contraction of the relations ri and r j on G ^ A 1, A 2, f, A N , T h . X r + r can be derived by adding the ri -th and rj -th slices of tensor X G . ◆◆ Contract vertex: Given t i, t j ! A m and t i has relations with t j, where A m is an entity set. The graph tensor X t + t denotes contracting the entities t i and t j and removing all edges between the two vertexes. X t + t can be calculated by adding the t i -th and t j -th slices and setting the corresponding entries of X G to 0. ◆◆ Extract subgraph: The graph tensor X sub denotes extracting a subgraph of G ^ A 1, A 2, f, A N , T h, whose entities belong to entities set A m and relations belong to relations set A n. X sub is a subtensor of X G . i
i
j
i
j
j
i
N
Intragraph Operations The intragraph operations are used to update a nd modify vertexes or edges in a knowledge graph, including deleting, contracting, and extracting subgraphs,
Intergraph Operations The intergraph operations are used to integrate two or more graphs, including union, intersection, and product, and so on. More formalized definitions are given as follows: ◆◆ Graph tensor union: Given two knowledge graphs G 1 ^ A 1, A 2, f, A N , T1 h and G 2 ^ B 1, B 2, f, B N , T2 h, whose representation tensors are X G and X G , respectively. The union of the two graph can be obtained by graph tensor union operation, i.e., X G ,t X G = X G , where , t denotes implementing union operation on two entries at the same index of the two graph tensor. X G is the representation tensor of the union graph with size I 1C # I 2C # I CN , and I Cn is the length of set C n = A n , B n. ◆◆ Graph tensor intersection: Given two knowledge graphs G 1 and G 2, their representation tensors are X G and X G , as described earlier. The intersection of the two graphs can be calculated by the graph tensor intersection, i.e., X G +t X G = X G , where + t denotes executing intersection operation on two entries at the same index of the two graph tensor. X G is the representation tensor of the intersected graph with size I 1C # I 2C # I CN , and I Cn is the length of set C n = A n + B n. ◆◆ Graph tensor product: Given two graphs G 1 and G 2, the product of them can be obtained by graph tensor product, i.e., X G #t X G = X G , where # t stands for the filling 1s to X G according to the indexes in G 1 and G 2 . X G is the representation tensor of the product 1
2
1
Table 1. The graph tensor operations and definitions. Type
Graph Tensor Operation
Definition
Intragraph
Extract subgraph
X sub
Delete edge
X G -r
Delete vertex
X G -t
Contract edge
X r +r
j
Contract vertex
X t +t
j
Graph tensor union
X G ,t X G
2
Graph tensor intersection
X G +t X G
2
Graph tensor product
X G #t X G
2
Graph tensor decomposition
X G +t X G
1
i
1
1
1
1
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE Apri l 2022
2
I
I
1
P
2
P
34
U
2
1
Intergraph
2
U
i
j
2
P
graph with size I 1C # I 2C # I CN , and I Cn is the length of set C n, which is the Cartesian product of A n and B n. ◆◆ Graph tensor decomposition: The decomposition of G ^ A 1, A 2, f, A N , T h means decomposing G into two edge disjoint subgraphs, which can be denoted by X G = X G +t X G , where X G and X G do not have common 1 entries at the same indexes, where +t denotes the graph tensor summation. Note that graph tensor decomposition is different from normal tensor decomposition, such as Tucker, CP, etc., since it aims at finding two subtensors of the graph tensor G. We present an example of the contract vertex operation in Figure 3. First, the knowledge graph G can be represented by the tensor X G with size 4 # 4 # 2. Then, by performing the addition operation on the 2-th and 3-th along the two entity orders, we can obtain the tensor X e + e , which corresponds to the knowledge G e + e . D
1
2
1
2
2
2
3
3
denotes the subject slice and E i ^:, :, 12 h denotes the object slice. The iterative calculation process of the arbitrar y m + 1^ m 2 = 2 h hop relation path tensor is as follows:
P m + 1 = X G #1 P mM (2)
where P mM is the unfolding matrix of tensor P m along the second order. m times 1-mode product between the knowledge graph representation tensor and the unfolding matrix can obtain the m + 1 hop relation path tensor. The relation path tensor P is inter pretable: arbitrar y P m + 1 ^ rm, j, rm + 1 h = 1 in P m + 1 indicates the relation path r r ei g $ $ g e j . In summary, the steps to reason the relationship between two arbitrary entities e i and e j are shown as follows: ◆◆ first constructing the representation tensor of the entity e i ◆◆ then, inferring the relation path Pe ,e = " r1, r2, f, rM , based on the relation path tensor as described in (1) and (2) ◆◆ taking the corresponding relation slice in XG according to the relation path and multiplying them successively: R P = P mM =1 X G ^:, :, m h. ◆◆ finally, finding the relationship that satisfies the conditions " r ! R ; X G ^i, j, r h = 1, 6R P^i, j h = 1 ,, which is the inferred possible relation set between e i and e j . m
m+1
Tensor-Based Knowledge Reasoning After performing representation and fusion operations, we conduct reasoning tasks over the integrated knowledge graph using tensor operations. Specifically, we develop a tensor operation-based link (relation) prediction method. First, we use the third-order tensor X G ! R N # N # N to represent the knowledge graph according to the description in the “Tensor Representation” section, where N E and N E are the length of the entity set E and the relationship set R, respectively. A Case Study We utilize E i ! R N # N # 2 to denote the entity representaTo further illustrate the effectiveness of the proposed methods, we demonstrate how to apply the tensor-based tion tensor of e i, where the first order tensor represents knowledge representation, fusion, a nd rea soning the relationship, the second order represents the entity, approaches. To improve readability, we present an and the two slices of the third order represent the subject easy-to-understand example in this section. Our model slice and the object slice, respectively, which indicate i s a l so appl ic able t o more complex a nd l a rger whether the entity e i is the subject or the object in triples. The elements in the entity representation tensor indicate the existence of triples containing e i . Knowledge graphs are Xe2+e3 XG directed graphs, and the relationship 1 direction between entities has specific Relation semantic information, so the relationship 1 Relation 1 direction cannot be ignored. In our entity Entity 1 Entity representation tensor, we effectively 1 retain the relationship direction informa1 Entity Entity 1 tion in the triples by treating the subject and the object as different slices. In subsequent inference calculations, different Ge2+e3 slices can be taken to determine the direcG tion of the relationship. r2 e2 e3 e2/3 Then, the 2-hop relation path tensor of r1 r1 entity e i can be obtained as follows: i
E
R
E
R
j
E
r1
r1
P m = X G #1 E i ^:, :, b h (1)
e1
where m = 2, P m has size N R # N E # N R, E i ^:, :, b h, and ^b = 1, 2 h denotes the slice on the third order of E i, i.e., E i ^:, :, 1 h
r2
e4
e1
r2
e4
Figure 3. An example of contract vertex operation.
Ap ri l 2022
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE
35
r1
r2
e1
r1
e3
e3
r2
r1
e1
e5
e1
e2
e3
e5
e4
r2
1
e1
e2
e3
e5
e4
r3
e4
e5
e5
e1
e2
e3
e4
e5
e1 1
e2
e3
e4
e5
r3
(c)
e2
e3
r2
e3
?
e1
e2
r1
r2
(b)
1
e1
r1
r3
(a)
e4
e2
e4
e2
1
e3 1
e4 e5
Slice r1
Slice r2
Slice r3
(d)
Figure 4. An example of tensor-based knowledge representation and fusion: (a) knowledge graph G 1,
(b) knowledge graph G 2, (c) fused knowledge graph G, and (d) fused representation tensor XG .
(e1, r1, e2)
R S
e1
e2
r1
e3
e4
e5
e1
O
e2
e3
e4
e5
r1
1
r2
r2
r3
r3 Slice Subject
E
Slice Object (a) r1 r2 P : e1 e3
R1_hop r1
e1
e2
e3
e4
e5
r1
e1
e2
e3
e4
e5
r1
r1
r1
r2
r2
r2
r3
r3
r3
R2_hop Slice r1
1
R2_hop Slice r2
e1
e2
e3
e4
e5
r1
R2_hop Slice r3
(b)
Figure 5. An entity representation tensor and relation path tensor: (a) entity representation tensor E 1 and
(b) relation path tensor P.
36
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE Apri l 2022
E
knowledge graphs. Without loss of generality, we do not specify any specific entity and relation, but represent them by e and r. In the illustrated example, the reasoning task is set to reason about the relationship between entities e 1 and e 3 .
Knowledge graphs are directed graphs, and the relationship direction between entities has specific semantic information, so the relationship direction cannot be ignored.
Tensor-Based Knowledge Representation and Fusion First, according to the proposed method in the “Tensor Representation” section, the knowledge graph G 1 and G 2 in Figure 4(a) and Figure 4(b) are represented by tensors X G ! R 3 # 3 # 2 and X G ! R 3 # 3 # 3. Then, the fused knowledge graph is obtained by graph tensor operations according to the “Intragraph Operations” section. In this example, we conduct graph tensor union operation between X G and X G and obtain the fused representation tensor X G with size 5 # 5 # 3. The relation slices are shown in Figure 4(d). 1
2
1
2
Conclusion With the in-depth integration of cyber, physical, and social spaces, the data in the CPSS are growing explosively at unprecedented s p e e d , a nd t he r el a t ion s h ip between entities has also changed from a traditional single view to multiple views, which further accelerates the formation of highorder knowledge graphs with complex association relationships. How to effectively manage and process the knowledge from multisource-related data to further promote artificial intelligence decision making has become an important research task. This article proposes a set of integrated tensorbased knowledge analysis methods, including the
E
Tensor-Based Knowledge Reasoning The entity representation tensor of e 1 is shown in Figure 5(a). In our example, e 1 is contained in the triple ^e 1, r1, e 2 h a nd a c t s a s s u bje c t ; t hu s , t he ele m e n t E ^1, 2, 1 h = 1. A fter per for ming X G #1 E ^:, :, 1 h according to (1), we obtain the 2-hop relation path tensor P with size 3 # 5 # 3, which is presented in Figure 5(b). The first order denotes the 1-hop relation and the third denotes the 2-hop relation as marked in Figure 5(b). Therefore, the element P ^1, 3, 2 h = 1 indicates the relar r tion path e 1 " " e 3 . The more detailed calculation process about the relation path can be seen in the first example in Figure 6. We also show more examples in Figure 6 to make the computation of relation paths easier to understand. We take the r1 and r2 relation slices of the knowledge graph tensor X G on the r r basis of the relation path e 1 " " e 3 . As illustrated in Figure 7, R P = R 1 # R 2 and the entity pairs in the matrix have the same relation path, which implies that these entit y pa irs a re most likely to have t h e sa me relationship. Specif ica lly, R P^1, 3 h = 1 and R P^3, 5 h = 1 mean that we can infer the relationship between the target entities e 1 and e 3 through the entities e 3 and e 5 . Finally, from X G ^3, 5, 3 h = 1, we 1
infer that the relation between entities e 1 and e 2 may be r3 .
XG
r1
× e2 ×
r4
P
(e2, r2, e3)
(e2, r2, e3)
e2 r1
×
=
r1 r2 e1 → e2 → e3
r1
r2
r2
e3
r4 r2 e1 → e2 → e3
= r 4 r2
(e4, r1, e2)
r2
e3 =
r1
r1
e4
Relation Path
r1 r1 e1 → e4 → e2
r1
e2
2
1
×
r4
(e4, r3, e2) r3
e4
r4 r3 e1 → e4 → e3
= r4 r3
e2
Figure 6. Some examples of calculation of relation paths.
2
r1 e1 e2 e3 e4 e5 e1 1 e2 e3
1
r2 e1 e2 e3 e4 e5 e1 ×
e2
1
=
e3
e4
e4
e5
e5
R1 = XG (:, :, 1)
e1 e2 e3 e4 e5 e1
1
e2 e3
1
1
e4 e5
R1 = XG (:, :, 2)
RP
Figure 7. The multiplication of relation slices.
Ap ri l 2022
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE
37
representation, fusion, and reasoning of heterogeneous knowledge graphs, which are generalized and practical to high-order knowledge graphs. What’s more, the tensorbased knowledge fusion and reasoning framework and approaches proposed in this article can be applied to a wide range of real-world scenarios, such as semantic search, knowledge question and answer recommendation systems, and so on. Furthermore, we want to explain that this article provides tensor-based ideas and frameworks for knowledge reasoning and fusion. Tensors have sufficient mathematical theories and adaptable operations, from which we will further explore knowledge fusion and reasoning methods based on tensors in future work.
Economics. He is an M.Sc. degree student in computer science and technology at Huazhong University of Science and Technology, Wuhan, 430074, China. His research interests include knowledge graph and natural language processing. Xia Xie ([email protected]) earned her Ph.D. degree from Huazhong University of Science and Technology in 2006. She is a professor at Hainan University, Haikou, 570000, China. Her research interests include knowledge graph, data mining, and big data. References [1] J. Yang, L. T. Yang, Y. Gao, H. Liu, and X. Xie, “An incremental Boolean tensor factorization for knowledge reasoning in AIoT,” IEEE Trans. Ind. Informat., vol. 8, no. 5, pp. 3367–3376, 2021, doi: 10.1109/TII.2021.3100978.
Acknowledgment Jing Yang is the corresponding author of this article.
[2] K.-C. Li, H. Jiang, L. T. Yang, and A. Cuzzocrea, Big Data: Algorithms, Analytics, and Applications. Boca Raton, FL, USA: CRC Press, 2015. [3] X. Wang, L. T. Yang, H. Liu, and M. J. Deen, “A big data-as-a-service framework:
About the Authors Jing Yang ([email protected]) earned her B.E. degree in communication engineering from Harbin University of Science and Technology. She is a Ph.D. degree student at Huazhong University of Science and Technology, Wuhan, 430074, China. Her research interests include knowledge graph, representation learning, and data mining. Laurence T. Yang ([email protected]) earned her B.E. degree in computer science and technology and B.Sc. degree in applied physics both from Tsinghua University, and her Ph.D. degree in computer science from the University of Victoria. He is a professor at Huazhong University of Science and Technology, Wuhan, 430074, China, and at St. Francis Xavier University, Antigonish, NSB2G 2W5, Canada. His research interests include parallel and distributed computing, embedded and ubiquitous/pervasive computing, and big data. He is a Fellow of IEEE. Yuan Gao ([email protected]) earned his B.E. degree from Wuhan University of Science and Technology. He is currently studying for his Ph.D. degree in the School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, 430074, China. His research interests include data mining, parallel and distributed computing, security computing, and deep learning. Huazhong Liu ([email protected]) earned his B.Sc. degree in computer science from Jiangxi Normal University, his M.Sc. degree in computer science from Hunan Normal University, and his Ph.D. degree in computer science from Huazhong University of Science and Technology. He is a professor at Hainan University, Haikou, 570000, China. His research interests include big data, cloud computing, Internet of Things, and scheduling optimization. His research has been supported by the National Natural Science Foundation of China. Hao Wang ([email protected]) earned his B.E. degree from the Central University of Finance and
38
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE Apri l 2022
State-of-the-art and perspectives,” IEEE Trans. Big Data, vol. 4, no. 3, pp. 325–340, Sep. 2018, doi: 10.1109/TBDATA.2017.2757942. [4] D. P. Vidyarthi, B. K. Sarker, A. K. Tripathi, and L. T. Yang, Scheduling in Distributed Computing Systems: Analysis, Design and Models. Springer Science & Business Media, 2008. [5] H. Zhu and I. Bayley, “Discovering and investigating cyberpatterns: The road map to link data analytics with reusable knowledge,” IEEE Syst., Man, Cybern. Mag., vol. 4, no. 3, pp. 14–22, Jul. 2018, doi: 10.1109/MSMC.2018. 2821200. [6] X. Dong et al., “Knowledge vault: A webscale approach to probabilistic knowledge fusion,” in Proc. 20th ACM SIGKDD Int. Conf. Knowl. Disc. Data Min., 2014, pp. 601– 610, doi: 10.1145/2623330.2623623. [7] F. Zhang, N. J. Yuan, D. Lian, X. Xie, and W.-Y. Ma, “Collaborative knowledge base embedding for recommender systems,” in Proc. 22nd ACM SIGKDD Int. Conf. Knowl. Disc. Data Min., 2016, pp. 353–362, doi: 10.1145/2939672.2939673. [8] G. Fortino, C. Savaglio, G. Spezzano, and M. Zhou, “Internet of Things as system of systems: A review of methodologies, frameworks, platforms, and tools,” IEEE Trans. Syst. Man Cybern., vol. 51, no. 1, pp. 223–236, Jan. 2021, doi: 10.1109/ TSMC.2020.3042898. [9] H. Zhu and I. Bayley, “Discovering and investigating cyberpatterns: The road map to link data analytics with reusable knowledge,” IEEE Syst., Man, Cybern. Mag., vol. 4, no. 3, pp. 14–22, 2018, doi: 10.1109/MSMC.2018.2821200. [10] K. Guo, Y. Lu, H. Gao, and R. Cao, “Artificial intelligence-based semantic Internet of Things in a user-centric smart city,” Sensors, vol. 18, no. 5, pp. 1341–1363, Apr. 2018, doi: 10.3390/s18051341. [11] Y. Li, C. Chen, M. Duan, Z. Zeng, and K. Li, “Attention-aware encoder-decoder graph neural networks for heterogeneous graphs of things,” IEEE Trans. Ind. Informat., vol. 17, no. 4, pp. 2890–2898, 2021, doi: 10.1109/TII.2020.3025592. [12] T. G. Kolda and B. W. Bader, “Tensor decompositions and applications,” J. Soc. Ind. Appl. Math, vol. 51, no. 3, pp. 455–500, Aug. 2009, doi: 10.1137/ 07070111X. [13] I. V. Oseledets, “Tensor-train decomposition,” J. Sci. Comput., vol. 33, no. 5, pp. 2295–2317, Jun. 2011, doi: 10.1137/090752286. [14] J. Yang, L. T. Yang, H. Wang, Y. Gao, H. Liu, and X. Xie, “Tensor graph attention network for knowledge reasoning in Internet of Things,” IEEE Internet Things J., early access, 2021, doi: 10.1109/JIOT.2021.3092360.
Bitcoin Price Prediction in a Distributed Environment Using a Tensor Processing Unit A Comparison With a CPU-Based Model by Mohd Hammad Khan, Devdutt Sharma, N. Narayanan Prasanth, and S.P. Raja price-prediction models. This article proposes a Bitcoin price-prediction model using a long short-term memory (LSTM) network in a distributed environment. A tensor processing unit (TPU) has been used to provide the distributed environment for the model. The results show that
©SHUTTERSTOCK.COM/PARILOV
B
itcoin is the world’s most traded cryptocurrency and highly popular among cryptocurrency investors and miners. However, its volatility makes it a risky investment, which leads to the need for accurate and fast
Digital Object Identifier 10.1109/MSMC.2021.3118893 Date of current version: 25 April 2022
2333-942X/22©2022IEEE
Ap ri l 2022
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE
39
the TPU-based model performed significantly better than a conventional CPU-based model. Background Bitcoin revolutionized the technical and financial world in 2009. It was the world’s first decentralized cryptocurrency. The specialty of Bitcoin is its algorithm, which limits its quantity to 21 million and ensures that all of the payments are irreversible and anonymous. In the last 10 years, advancements in GPU capabilities have made Bitcoin mining very popular. While Bitcoin has generated large profits for some investors, frequent fluctuations in its price make it a high-risk investment. Bitcoin’s volatility can be attributed to factors like its small market size, low liquidity, and public perception [1]. Because of its volatility, investors are wary of Bitcoin and want to find methods to reduce the risks associated with Bitcoin investment. To make it less risky, researchers have developed price-prediction models for Bitcoin. In recent years, with increases in computational power, machine learning algorithms have been used to develop price-prediction models [2]. Machine learning-based price-prediction models are generally the most accurate prediction models and provide a fair idea about the direction of the price. However, to train machine learning models, large volumes of data are required. This is especially true in the case of price-prediction models, where the price is dependent on the volume traded. Because of the size of the data sets, machine learning models can take considerable time to train, and the long training time can cause delays in deployment of the models in the real world. An effective way to reduce the training time is by parallelizing or distributing the training process among several processing units that can execute independent branches of code. However, while parallelizing the training process, various factors like synchronization and fault tolerance must be kept in mind. Hence, it can be quite tedious to develop custom frameworks for distributed machine learning. To counter this problem, we propose a Bitcoin priceprediction model that utilizes distributed machine learning to speed up the process of training and testing. An LSTM network architecture is used for building the model, while a TPU is used for distributing the training process. The main advantage of the proposed method is that it distributes the training process without requiring any significant changes in the code. A TensorFlow library is used for making use of the TPU for distributed machine learning. TensorFlow is a highly popular machine learning library known for its data visualization and pipelining capability. It has a wide range of machine learning algorithms that enable users to develop and deploy machine learning models as well as neural networks. TensorFlow is highly parallel and provides modules for distributed learning. TensorFlow’s tf.distribute is an application programming interface (API) that allows users to distribute the training across multiple computational devices and implement 40
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE Apri l 2022
different strategies for algorithm distribution [3]. Training can be distributed across multiple GPUs or TPUs depending upon the user’s requirements. Distributed Machine Learning Using a TPU and an LSTM Network The TPU The TPU is an application-specific integrated circuit designed for handling large machine learning workloads. It was announced by Google in 2016 and opened up for thirdparty use in 2018. It can carry out a large number of low-precision computations. When used in training large neutral networks, TPUs can minimize the time to accuracy [4]. Working of the TPU The TPU has been developed specifically for machine learning as a matrix processor that enables it to carry out large volumes of additions and multiplications at very high speeds. While executing calculations, the TPU loads a matrix of adders and multipliers with parameters retrieved from memory. Subsequently, data are loaded and multiplication operations are executed. After each operation, the result is passed over to the next multipliers, simultaneously taking a summation. Hence, the output is the total of all multiplication results that occurred between the parameters and data. All of the calculations and data passing are performed without accessing the memory again [4]. The LSTM Network One of the best models for time-series forecasting is an LSTM network, which is a type of recurring neural network that is capable of learning long-term dependencies. The LSTM structure consists of a chain format that includes a neural network and cells (which are the memory blocks). The LSTM memory cell has three types of gates: input, output, and forget gates. The forget gate is used to remove information from a memory cell that is no longer useful to store. New useful information is added to the cell using an input gate. The output gate is used to release the useful information stored in the cell as output. Related Works Significant research has been conducted to find relationships between the Bitcoin price and social or economic factors, like public perception, regulations, economic policies, and stakeholder behavior. Studies have shown that the volatility of Bitcoin is strongly influenced by security breaches, regulation-related news, and the leverage effect [5], [6]. Additionally, findings show that Bitcoin miners often exploit the Bitcoin ecosystem to maximize their profits [7]. Furthermore, network cohesion has been shown to have a negative relationship with Bitcoin price movements with regard to social media discussions [8]. Generally, it is difficult to quantify such factors for developing accurate
price-prediction models. However, with access to a larger variety and volume of Bitcoin-related data and the increase in computation power, several techniques have been employed for predicting the Bitcoin price. Sentiment analysis has helped in establishing correlations between sentiments and Bitcoin price. Researchers have demonstrated that the Bitcoin price can be predicted by performing sentiment analysis on news articles [9]. Sentiments expressed on social media platforms and networks have also proven to be strong indicators of Bitcoin price movements, with polarity indicating the direction of the price movement [10], [11]. Similarly, a wide variety of conventional machine learning algorithms along with neural networks has been used by researchers to create priceprediction models. Some of the algorithms used include Recurrent Neural Network + LSTM + Autoregressive Integrated Moving Average [2] and Convolutional Neural Network + LSTM + Gated Recurrent Unit [12]. To achieve better performance than that of conventional machine learning models, ensemble machine learning has been employed [13]. Furthermore, the usage of stacking-ensemble learning and variational mode decomposition has given better results than those of traditional ensemble learning models [14]. While some research works have shown the effect of a processor and parallelism on the training time of machine learning models [2], [15], the idea of employing distributed machine learning with the help of specialized processors like TPUs to develop price-prediction models has not been explored in depth until now. However, in many other domains, distributed machine learning has proven effective in reducing training time and latency. For example, a TPU-based high-speed object tracking and prediction model has demonstrated better performance than other architectures and accelerators [16]. In addition, TPU-trained models were found to be highly efficient in terms of computational time when trained for facial emotion recognition [17] and magnetic resonance image reconstruction [18]. Hence, to check the efficiency of a TPU in accelerating the training process of an LSTM-based price-prediction model, we employ it to provide a distributed environment for the proposed model. In this article, a TPU has been used to evaluate its efficiency in accelerating the training process of LSTM-based price-prediction models. Methodology In the proposed model, univariate time-series forecasting is implemented on a CPU-based system using an LSTM network architecture. The same method of time-series forecasting is then implemented on a TPU-based system by employing distributed machine learning. Initially, the data are collected and preprocessed. The division of the data into a training and a testing set is carried out after preprocessing. Subsequently, an appropriate machine learning algorithm is selected for the creation of a model (LSTM in
this case). The created model is trained using the training data set. Finally, the performances of the CPU- and TPUbased models are compared. Data Preparation The data set for training and testing the model was acquired from Quandl [19]. Quandl is an online currency platform and marketplace from which data about various currencies can be acquired using a Quandl API. The data set is made up of Bitcoin prices on 2,473 days (from 24 January 2014 to 1 November 2020). A univariate time-series forecasting technique is used for the prediction of future Bitcoin prices. For more accurate predictions, the weighted price of Bitcoin is used. The weighted price gives more insight into the movements and fluctuations as it provides the average price of the Bitcoin throughout the day. In the proposed method, the weighted Bitcoin price of the previous days is used to predict the weighted price of Bitcoin for the upcoming day. The weighted price, W(P), is calculated by summing the product of the traded volume and the corresponding trading price. Subsequently, the summation is divided by the total volume traded for the day as given in (1). Volume denotes the total amount of asset (Bitcoin in this case) bought or sold in a particular time frame. The weighted price is calculated for each day that appears in the data set.
W^ P h =
| ^ Volume Traded # Trading Price h Total Volume Traded in the Day . (1)
After the weighted price calculations, data preprocessing is carried out to normalize the data set. Normalization of the data set fits the data values into a common scale with no effect on the range of difference that exists in the data. Minmax normalization has been utilized in the proposed model, where each data point is mapped to a particular value between zero and one. The largest data point is mapped to one, while the smallest data point is mapped to zero. To obtain the normalized value z, the difference between the data value x and the smallest data value min(x) is divided by the difference between largest data value max(x) and the smallest data value min(x), as can be inferred from the following:
z=
x - min ^ x h . (2) max ^ x h - min ^ x h
After normalization of the data set, it is split into training and testing data; the training data are used to train the model, and the testing data are used to measure the model performance. For the proposed model, 70% of the data set is used as training data and 30% as testing data. Parallelization of the Model and TPU Usage Following the preprocessing of the data set, the model is parallelized and run on a TPU-based system. This is done to Ap ri l 2022
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE
41
check if the TPU provides any acceleration to the computations that are carried out as part of machine learning. Simultaneously, the model is also run on a CPU-based system that provides a benchmark to evaluate the performance of the TPU-based system. The TPU uses quantization to scale back the execution time. Quantization is an optimization technique that uses an 8-bit integer to approximate an arbitrary value that is between a preset minimum and a maximum value. Therefore, rather than using 16-bit or 32-bit floating-point operations, 8-bit integer operations are used for prediction-related calculations using quantization [20]. Hence, quantization significantly reduces the computational requirements by maintaining a precision similar to that of floating-point calculations while using integer-based calculations.
Bitcoin Price Prediction
Bitcoin Value (Weighted Price)
0.8 0.7 0.6 0.5 0.4 0.3 0.2
Real Bitcoin Value Predicted Bitcoin Value
0.1 0
0
100
200
300 400 500 Time (Days)
600
700
Figure 1. The accuracy graph of the TPU-trained model.
Bitcoin Price Prediction
Bitcoin Value (Weighted Price)
0.8 0.7 0.6 0.5 0.4 0.3 0.2
Real Bitcoin Value Predicted Bitcoin Value
0.1 0
0
100 200 300 400 500 600 700 Time (Days)
Figure 2. The accuracy graph of the CPU-trained model. 42
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE Apri l 2022
For utilizing the TPU, a Google Colab notebook is used as a coding platform because, at present, the TPU can only be used within a Colab notebook. To run and process the code on the TPU, it is selected as the runtime hardware accelerator in the Colab notebook. Subsequently, the TPU is initialized using TPUClusterResolver. Furthermore, the name of the Cloud TPU is passed to the TPUClusterResolver to make use of Google Compute Engine. After selection and initialization, TensorFlow’s distribution functionality (tf.distribute) is used along with the TPU strategy to make the TPU ready for parallelization. The use of tf.distribute enables parallelization of the model without requiring any significant code modification. After creation and training, the performances of the sequential (CPU-trained) and parallelized (TPU-trained) model are evaluated and compared. The training time and mean-squared error are used as parameters for finding the efficiency and accuracy of both models. The training time indicates how fast the processor is able to complete the training process of the model and make it suitable for deployment. The mean-squared error provides the average of the squared difference between the actual and predicted values. It is a good indicator of a model’s ability to accurately predict values. Results and Discussion In the proposed model, an Intel Xeon processor (2.3 GHz) with 12.6-GB random-access memory (RAM) is utilized as the CPU and a TPU v2 with 12.6-GB RAM is used as the TPU. In addition, Google Colab notebook is used as the coding platform. Python 3 is used as the coding language. To incorporate the LSTM network architecture into the model, TensorFlow and Keras machine learning libraries are employed. After the creation of the LSTM model, it is trained separately on the CPU and TPU. While the CPU trains the model sequentially, the TPU employs distributed machine learning to train the model. The training times for the CPU-based sequential model and TPU-based parallelized model were 84 and 24.7 s, respectively. The mean-squared error values obtained for the CPU- and TPU-based models were 0.00044 and 0.0013, respectively. As can be inferred from Figure 1, the parallelized TPU-based model was able to correctly predict the direction of the Bitcoin price (rising or falling) with a high degree of accuracy. However, the prediction accuracy of the TPU-trained model was slightly lower than that of the CPU-trained model. This can be understood by comparing Figures 1 and 2, where the actual weighted Bitcoin price and predicted weighted Bitcoin price for all days in the testing data set can be seen. This is also confirmed by the mean-squared error values obtained, wherein the CPU-based model achieved smaller values than the TPU-based model. However, the para l lel i z ed T PU - t r a i ned model out per for med t he CPU-trained model in terms of computational time by
taking 70.5% less time to train the model. Moreover, the accuracy of the TPU trained model was adequate to be employed in a real-life application, which makes the tradeoff between training time and accuracy profitable. This means that the TPU is a more efficient option for training and testing different machine learning-based Bitcoin price-prediction models.
[4] “Concepts,” Google Cloud. https://cloud.google.com/tpu/docs/concepts (Accessed: May 29, 2021). [5] Š. Lyócsa, P. Molnár, T. Plíhal, and M. Širanˇová, “Impact of macroeconomic news, regulation and hacking exchange markets on the volatility of bitcoin,” J. Econ. Dyn. Control, vol. 119, p. 103,980, Aug. 2020, doi: 10.1016/j.jedc.2020.103980. [6] M. Yu, “Forecasting Bitcoin volatility: The role of leverage effect and uncertainty,” Physica A, Statistical Mech. Appl., vol. 533, p. 120,707, Nov. 2019, doi: 10.1016/ j.physa.2019.03.072.
Conclusion As the craze for cryptocurrency rages, better and more efficient methods to predict the price of Bitcoin will be needed. In this article, we have developed an LSTM-based Bitcoin price-prediction model using univariate time-series forecasting. Furthermore, we have parallelized the model using a TPU to reduce the training time. The parallelized model has shown an ability to reduce the training time by a significant percentage without adversely affecting the accuracy of the model. In addition, the parallelization of the model did not require any significant changes in the code. However, the proposed method employed only one machine learning algorithm: LSTM. Moreover, only two parameters were used to evaluate the performance of the sequential and distributed models. In the future, multivariate neural networks could be utilized to develop more complex Bitcoin price-prediction models. Additionally, a larger number of evaluation parameters could be used to gain further insight into the performance and efficiency of parallelized models. This could help provide a better understanding of the usability and efficiency of distributed machine learning and specialized processors, such as TPUs.
[7] B. Hou and F. Chen, “A study on nine years of bitcoin transactions: understanding real-world behaviors of bitcoin miners and users,” in Proc. IEEE 40th Int. Conf. Distrib. Comput. Syst. (ICDCS), 2020, pp. 1031–1043, doi: 10.1109/ ICDCS47774.2020.00091. [8] P. Xie, H. Chen, and Y. J. Hu, “Signal or noise in social media discussions: The role of network cohesion in predicting the bitcoin market,” J. Manage. Inf. Syst., vol. 37, no. 4, pp. 933–956, 2020, doi: 10.1080/07421222.2020.1831762. [9] W. Yao, K. Xu, and Q. Li, “Exploring the influence of news articles on bitcoin price with machine learning,” in Proc. IEEE Symp. Comput. Commun. (ISCC), 2019, pp. 1–6, doi: 10.1109/ISCC47284.2019.8969596. [10] G. Serafini et al., “Sentiment-driven price prediction of the bitcoin based on statistical and deep learning approaches,” in Proc. Int. Joint Conf. Neural Netw. (IJCNN), 2020, pp. 1–8, doi: 10.1109/IJCNN48605.2020.9206704. [11] O. Sattarov, H. S. Jeon, R. Oh, and J. D. Lee, “Forecasting bitcoin price fluctuation by twitter sentiment analysis,” in Proc. Int. Conf. Inf. Sci. Commun. Technol. (ICISCT), 2020, pp. 1–4, doi: 10.1109/ICISCT50599.2020.9351527. [12] A. Aggarwal, I. Gupta, N. Garg, and A. Goel, “Deep learning approach to determine the impact of socio economic factors on bitcoin price prediction,” in Proc. 12th Int. Conf. Contemporary Comput. (IC3), 2019, Noida, India, pp. 1–5, doi: 10.1109/IC3.2019. 8844928. [13] S. Lahmiri, R. G. Saade, D. Morin, and F. Nebebe, “An artificial neural networks based ensemble system to forecast bitcoin daily trading volume,” in Proc. 5th Int. Conf. Cloud Comput. Artif. Intell., Technol. Appl. (CloudTech), 2020, pp. 1–4, doi: 10.1109/
About the Authors Mohd Hammad Khan (mohdhammad.khan2018@ vitstudent.ac.in) is a student in the School of Computer Science and Engineering, Vellore Institute of Technology, Vellore 632 014, Tamil Nadu, India. Devdutt Sharma ([email protected] .in) is a student in the School of Computer Science and Engineering, Vellore Institute of Technology, Vellore 632 014, Tamil Nadu, India. N. Narayanan Prasanth ([email protected]) is an associate professor in the School of Computer Science and Engineering, Vellore Institute of Technology, Vellore 632 014, Tamil Nadu, India. S.P. Raja ([email protected]) is an associate professor of School of Computer Science and Engineering, Vellore Institute of Technology, Vellore 632 014, Tamil Nadu, India.
10.1109/ICE/ITMC49519.2020.9198366.
References
tion on TPUs,” in Proc. IEEE High Perform. Extreme Comput. Conf. (HPEC), 2020,
CloudTech49835.2020.9365913. [14] R. G. da Silva, M. H. Dal Molin Ribeiro, N. Fraccanabbia, V. C. Mariani, and L. dos Santos Coelho, “Multi-step ahead bitcoin price forecasting based on VMD and ensemble learning methods,” in Proc. Int. Joint Conf. Neural Netw. (IJCNN), 2020, Glasgow, U.K., pp. 1–8, doi: 10.1109/IJCNN48605.2020.9207152. [15] S. C. Purbarani and W. Jatmiko, “Performance comparison of bitcoin prediction in big data environment,” in Proc. Int. Workshop Big Data Inf. Security (IWBIS), 2018, pp. 99–106, doi: 10.1109/IWBIS.2018.8471691. [16] J. Sengupta, R. Kubendran, E. Neftci, and A. Andreou, “High-speed, real-time, spike-based object tracking and path prediction on google edge TPU,” in Proc. 2nd IEEE Int. Conf. Artif. Intell. Circuits Syst. (AICAS), 2020, Genova, Italy, pp. 134–135, doi: 10.1109/AICAS48895.2020.9073867. [17] R. D’Agostino and T. Schmidt, “EMOTIONET: A multi-convolutional neural network hierarchical approach to facial and emotional classification using TPUs,” in Proc. IEEE Int. Conf. Eng., Technol. Innov. (ICE/ITMC), 2020, Cardiff, U.K., pp. 1–4, doi: [18] T. Lu, T. Marin, Y. Zhuo, Y.-F. Chen, and C. Ma, “Accelerating MRI reconstruc-
[1] R. Shor, “What causes volatility in Bitcoin?” FXCM, 2017. https://www.fxcm.com/
Waltham, MA, USA, pp. 1–9, doi: 10.1109/HPEC43674.2020.9286192.
markets/insights/what-causes-volatility-in-bitcoin/ (Accessed: Sep. 26, 2021).
[19] “Bitcoin market price USD - Quandl,” Nasdaq Data Link. https://www.quandl.com/
[2] S. McNally, J. Roche, and S. Caton, “Predicting the price of bitcoin using machine
data/BCHAIN/MKPRU-Bitcoin-Market-Price-USD (Accessed: Mar. 13, 2021).
learning,” in Proc. 2018 26th Euromicro Int. Conf. Parallel, Distrib. Netw.-based Pro-
[20] “An in-depth look at Google’s first tensor processing unit (TPU),” Google Cloud,
cess. (PDP), pp. 339–343, doi: 10.1109/PDP2018.2018.00060.
2021. https://cloud.google.com/blog/products/ai-machine-learning/an-in-depth-look-at
[3] “Module: Tf.distribute | TensorFlow Core v2.8.0,” TensorFlow, 2021. https://www.
-googles-first-tensor-processing-unit-tpu (Accessed: Sep. 28, 2021).
tensorflow.org/api_docs/python/tf/distribute (Accessed: Sep. 26, 2021).
Ap ri l 2022
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE
43
Society News
by Haibin M.E. El-Hawary Zhu
Five Members Elected to the SMCS Board of Governors
D
r. Edward Tunstel, senior past president of the IEEE Systems, Man, and Cybernetics Society (SMCS) and chair of the 2021 Nominations Committee, announced that the following members have been elected to the SMCS Board of Governors (BoG) for three-year terms (1 January 2022–31 December 2024): ◆◆ Plamen Parvanov Angelov ◆◆ Yaoping Hu ◆◆ David Mendonça ◆◆ Ching-Chih Tsai ◆◆ Haibin Zhu. The IEEE received ballots from close to 20% of SMCS members. The following biographies are members of the new board. Plamen Parvanov Angelov Plamen Par vanov Angelov (a Life Fellow of IEEE) (M.Eng. degree 1989, Ph.D. deg ree 19 9 3, a nd D.Sc. degree 2015) is also a Fellow of the Institution of Engineering and Technology (IET), the European Laboratory for Learning and Intelligent Systems, and the Higher Education Academy of the U.K. He is governor of the SMCS and of the International Neural Networks Digital Object Identifier 10.1109/MSMC.2022.3150525 Date of current version: 25 April 2022
44
Society (INNS), and has been its vice president for two terms. He holds a personal chair in intelligent systems at Lancaster University, United Kingdom. He has authored or coauthored 370+ peer-reviewed publications in leading journals, peer-reviewed conference proceedings, six patents, and three research monographs (by Wiley, 2012, and Springer, 2002 and 2019) cited more than 12,300+ times with an h-index of 58. He is the founding director of the Lancaster Intelligent, Robotic and Autonomous Systems Research Center (www. lancaster.ac.uk/lira), which includes more than 60 academics across 15 departments from all faculties of the university. He has an active research portfolio in the area of computational intelligence and machine learning and internationally recognized results about online and evolving learning and explainable artificial intelligence (AI). Prof. Angelov leads numerous projects (including several multimillion pounds) funded by U.K. research councils, the European Union, industry, and the U.K. Ministry of Defence. His research was recognized by the 2020 Dennis Gabor Award “for outstanding contributions to engineering applications of neural networks,” the Engineer Innovation and Technology 2008 Special Award, and the award “for outstanding services” (2013) by
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE Apri l 2022
IEEE and INNS. He is also the founding co-editor-in-chief of the journal Evolving Systems and associate editor of several leading international scientific journals, including IEEE Transactions on Fuzzy Systems (impact factor 12.03), IEEE Transactions on Cybernetics (impact factor, 11.47), IEEE Transactions on Artificial Intelligence, and several other journals, such as Fuzzy Sets and Systems and Soft Computing, among others. He has given more than a dozen plenary and keynote talks at high-profile conferences. Prof. Angelov was general cochair of a number of high-profile conferences. He was also a member of the International Program Committee of 100+ international conferences (primarily IEEE events). More details can be found at www.lancs.ac.uk/ staff/angelov. Yaoping Hu Yaoping Hu (a Member of IEEE), a professor of software engineering, is a leading expert in human– computer interaction within virtual environments based on advent computer-based virtual/ extended reality (XR) technologies. Her research activities embrace pattern extraction/modeling of spatiotemporal data, 3D visualization and haptics for user interaction, and multiuser collaborative interaction. These activities have generated methodologies/knowledge essential for visual and interactive analytics of complex data and created XR systems for petroleum, medical, and civil applications. Her scientific acumen is exemplified by secured major grants/contracts as lead investigator, international and industrial research collaborations, numerous transaction/journal papers published
by the prestigious IEEE Press and MIT Press, and several best paper/ poster awards at flagship IEEE international conferences. Dr. Hu served as vice chair of finance of the IEEE Robotics and Automations Society Technical Committee on Haptics. She is a member of the SMCS Technical Committee on Human–Computer Interaction and a member of the IEEE Standards Association P2731 Working Group—Unified Terminology for Brain–Computer Interfaces. David Mendonça David Mendonça (a Senior Member of IEEE) is a sen ior principal researcher at the MITRE Corporation. His research interests are in human–machine systems and decision support technologies. He was previously a professor of industrial and systems engineering at Rensselaer Polytechnic Institute (RPI). He has also held positions at the U.S. National Science Foundation (NSF) and the New Jersey Institute of Technology. He has published extensively in the fields of human–machine systems and human-centered computing, with a focus on computer-based support for human cognition in high-stakes decision making. His work has been funded through grants from the NSF (including a CAREER award) and has involved extensive national and international collaborations. He is a member of the Cognitive Science Society and the Human Factors and Ergonomics Society. He has served in both organizing and program committee capacities for a wide variety of engineering and applied computing conferences throughout his career. He has significant experience in mentoring students and faculty, including through multiple NSF-funded mentoring projects. He holds degrees from RPI (Ph.D. degree), Carnegie Mellon University (M.S. degree) and the University of Massachusetts (B.A. degree). He is a longtime member of the SMCS, having served as associate editor of IEEE
Transactions on Human–Machine Systems and as a member of the SMCS BoG (2017–2020). Ching-Chih Tsai Ching-Chih Tsai (a Fellow of IEEE) is currently a life distinguished professor in the Department of Electrical Engineering, National Chung Hsing University (NCHU), Taichung, Taiwan, where he served as the 2012–2014 department chairman. He received his Ph.D. degree in electrical engineering from Northwestern University, Evanston, Illinois, in 1991. He has published more than 600 refereed journal and conference papers and seven patents in the fields of intelligent control and robotics, where he received many prestigious awards and honors from IEEE and numerous professional societies, and many best conference paper awards technically supported by IEEE. He served as the two-term president of the Chinese Automatic Control Society (CACS) from 2012 to 2015 and twoterm president of the Robotics Society of Taiwan (RST) from 2016 to 2019, and the dean of the R&D office, NCHU, in 2021. In recent years, he has served as associate editor of IEEE Transactions on Systems, Man, and Cybernetics: Systems, IEEE Transactions on Industrial Informatics, and International Journal of Fuzzy Systems. Moreover, he has served as the president of the International Fuzzy Systems Association (IFSA) since September 2021 and an IEEE SMCS BoG member and associate IEEE SMCS vice president of conferences and meetings since 2022. He has been elevated to Fellow of IEEE, IET, CACS, RST, and Taiwan Fuzzy Systems Association. Haibin Zhu Haibin Zhu (a Senior Member of IEEE) is a full professor and the coordinator of the Computer Science
Ap ri l 2022
Program, the founding director of the Collaborative Systems Laboratory, and a member of the University Budget Plan Committee and Arts and Science Executive Committee, Nipissing University, Canada. He is also an affiliate professor of Concordia University and an adjunct professor of Laurentian University, Canada. He received a B.S. degree in computer engineering from the Institute of Engineering and Technology, China (1983), and M.S. (1988) and Ph.D. (1997) degrees in computer science from the National University of Defense Technology (NUDT), China. He was chair of the Department of Computer Science a nd Mathematics, Nipissing University, Canada (2019–2021), a visiting professor and a special lecturer in the College of Computing Sciences, New Jersey Institute of Technology, USA (1999–2002), and a lecturer, an associate professor, and a full professor at NUDT (1988–2000). He has completed (published or in press) more than 200 research works, including 30+ IEEE transactions articles, six books, five book chapters, four journal issues, and four conference proceedings. He is a fellow of the International Institute of Cognitive Informatics and Cognitive Computing, a senior member of the Association for Computing Machinery, a Senior Member of IEEE, a full member of Sigma Xi, and a life member of the Chinese Association of Science and Technology, USA. He is serving as cochair of the Technical Committee of Distributed Intelligent Systems, a member of the Systems Science and Engineering (SSE) Technical Activity Committee, the Conferences and Meetings Committee, and the Electronic Communications Subcommittee of the SMCS, editor-in-chief of IEEE Systems, Man, and Cybernetics Magazine, associate editor of IEEE Transactions on Systems, Man, and Cybernetics: Systems, IEEE Transactions on Computational Social Systems, Frontiers of Computer Science, and IEEE Canadian Review. He has been an active
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE
45
organizer for the annual IEEE International Conference on Systems, Man, and Cybernetics since 2003. He was associate vice president of the SSE in 2021. He is the founding researcher of role-based collaboration and adaptive collaboration and the creator of the Environment-Classes, Agents, Roles, Groups, and Objects (E-CARGO) model. His newly published monograph, E-CARGO and Role-Based Collaboration, can be found on https:// w w w.a ma zon.com /CA RGO -Role Ba s ed - Col l a bor a t ion -Model i ngProblems/dp/1119693063. The accompanying codes can be downloaded from Git Hub: https://github.com/ haibinnipissing/E-CARGO-Codes. He has offered 70+ invited talks, including keynote and plenary speeches on related topics, internationally, e.g., in Canada, the United States, China, the United Kingdom, Germany, Turkey,
Hong Kong, Macau, and Singapore. His research has been being sponsored by the National Sciences and Engineering Research Council, IBM, Department of National Defense Canada, Defense Research and Development Centre Canada, and Ontario Partnership for Innovation and Commercialization. He is the recipient of the Meritorious Service Award from the SMCS (2018), the Chancellor’s Award for Excellence in Research (2011), and two research achievement awards from Nipissing University (2006, 2012), the IBM Eclipse Innovation Grant Awards (2004, 2005), the Best Paper Award from the 11th International Society for Productivity Enhancement International Conference on Concurrent Engineering, the Educator’s Fellowship of Object-Oriented Programming, Systems, Languages, and Applications 2003, a second class National Award for Education
What + If = IEEE
Achievement (1997), and three First Class Ministerial Research Achievement Awards from China (1997, 1994, and 1991). His research interests include collaboration systems, complex systems, human–machine systems, computational social simulation, collective intelligence, multiagent systems, software engineering, and distributed intelligent systems. About the Author Haibin Zhu (haibinz@nipissingu. ca) is a full professor and the coordinator of the Computer Science Program, the founding director of the Collaborative Systems Laboratory, and a member of the University Budget Plan Committee and Arts and Science Executive Committee, Nipissing University, North Bay, Ontario, P1B 8L7, Canada.
420,000+ members in 160 countries. Embrace the largest, global, technical community. People Driving Technological Innovation.
ieee.org/membership
#IEEEmember
46
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE Apri l 2022
Conference Reports
by Yuying Dong and Jiliang Luo
The 18th IEEE International Conference on Networking, Sensing and Control
T
he 18th IEEE Internation a l Conference on Network ing, Sen si ng a nd Cont rol (ICNSC 2021) was held virtually 3–5 December 2021. Its theme was “Industry 4.0 and Artificial Intel ligence.” The conference brought together both academic research ers a nd industr ia l practitioners to address new challenges, share solutions, and discuss future re search directions. ICNSC 2021 was originally sched uled to be held in Xiamen, a mod ern and international city in Fu jian, China. However, the event was moved to a virtual format due to the impact of the COVID-19 pandemic. We accepted 112 papers after peer review. The authors of the 112 ac cepted papers made excellent pre sentations virtually. It is worth not ing that the authors were from many countries, such as China, the Unit ed States, France, Australia, the United Kingdom, Canada, France, Ireland, Japan, and Italy. With the combined efforts of program and steering committees and the au thors who contributed excellent papers, ICNSC 2021 was held suc ce s sf u l ly as a multidisciplinar y international conference where scien tists, engineers, and students could meet and discuss their common in terests (Figure 1). Digital Object Identifier 10.1109/MSMC.2022.3149118 Date of current version: 25 April 2022
The technical program consisted of a panel session, three keynote speeches, the Best Paper Award selec tion session, the Best Student Paper Award selection session, and 21 tech nical sessions (including nine special sessions). The plenary session opened
with welcome addresses by Jianping Wu, president of Huaqiao University (Figure 2); Mengchu Zhou, ICNSC steering committee chair (Figure 3); and Jiliang Luo, ICNSC general chair (Figure 4). It was followed by three keynote speeches:
Figure 1. The opening ceremony of ICNSC 2021.
Figure 2. Huaqiao University President Jianping Wu delivers his
welcome address.
Ap ri l 2022
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE
47
1) Prof. Lixin Tang (Figure 5) from Northeastern University, China, presented “Data Analytics and Opti mization for Smart Industry.” This
talk discussed some interesting topics on system optimization and data analytics of production, logis tics, and energy in the steel industry.
Figure 3. ICNSC Steering Committee Chair Mengchu Zhou presents his
welcome address.
Figure 4. ICNSC General Chair Jiliang Luo gives his welcome address.
Figure 5. Prof. Lixin Tang presents his keynote speech. 48
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE Apri l 2022
2) Prof. Yuanqing Xia (Figure 6) from Beijing Institute of Technolog y, China, offered “Workflows Sched uling in Cloud Computing.” In this talk, Prof. Xia introduced the scor ing and dynamic hierarchyba sed, nondominated sor ting genetic algorithm II, a multiswarm coevolutionary-based hybrid opti mization algorithm, and a schedul ing algorithm. 3) Dr. Edward Tunstel (Figure 7) from Motiv Space Systems, Inc., USA, delivered “Toward a Nexus of Intel ligence and Interaction in Advan ced Robotics.” His talk explored advanced robotics in this context amid parallel advancements in arti ficial intelligence, machine learning, the Internet of Things, and so on with a focus on where robotic i ntelligence and human–robot interaction must meet to enable effective networking sensing and control collaboration. ICNSC feat u red n i ne specia l s e s sions and covered the follow ing topics: ◆◆ “Applying OR and Artificial Intel ligence for Solving Decision Prob lems in Industry 4.0.” ◆◆ “Intelligent and Energy-Efficient Applications in Edge Computing.” ◆◆ “Green and Intelligent Logistics and Transportation Systems.” ◆◆ “Intelligent and Learning Algo rithms for Scheduling Problems in Industry 4.0.” ◆◆ “Ripple Effect Management for the Supply Chain.” ◆◆ “Industry 4.0 and Artificial Intelli gence, From Theory to Industrial Case Study.” ◆◆ “Advances in Hybrid Model-DataBased Control Approaches in the Future Challenging Aerospace Problems.” ◆◆ “Monitoring, Control and Intel ligent Learning for Networked Systems.” ◆◆ “Petri Nets and Artificial Intelligence.” Accord i ng to statistics from the submissions (Figures 8 and 9), some topics received a great deal of research attention, such as Industry 4.0, robot systems, smart cars, artificial
Figure 6. Prof. Yuanqing Xia delivers his keynote
speech.
intelligence, supply chain, schedul ing methods, intelligent and learning methods, and optimization algorithms. During the technical sessions, participants had in-depth discus sions on them. Moreover, accord ing to the professional evaluation of five experts, the “A Virtual WaferBased Scheduling Method for DualArm Cluster Tools With Chamber Cleaning Requirements” paper won the best paper award in application, the “Distributed Adaptive Consen sus Via Event-Triggered Sampling: An Edge-Based Method” paper won the best paper award in theory, the “Neural Network and ANFIS Based Auto-Adaptive MPC for Path Track ing in Autonomous Vehicles” paper won the best student paper award in application, and the “Reducing Dif ferences Between Real and Realis tic Samples to Improve GANs” paper won the best student paper award in theory. ICNSC 2021 was hosted by the IEEE Systems, Man, and Cybernetics Society; Huaqiao University; Jiangxi University of Science and Engineer ing; and Zhejiang University; and co-hosted by automation commit tees of Xiamen, Fujian, and Jiangsu. This conference led to many engag ing conversations, numerous new friends, some exciting new research directions and very possibly, some papers that will be presented at fu ture conferences, including the 19th IEEE ICNSC, which will be held in Shanghai, China, 20–23 October 2022 and chaired by Prof. Qi Kang from Tongji University and Prof. Xin
Figure 7. Dr. Edward Tunstel gives his keynote speech.
30
20
28
20 13
10
Industry 4.0
Robot Systems
7
8
Petri Nets
Smart Cars
10
Artificial Intelligence
Supply Chain
Figure 8. The classification statistics of papers in application.
30
27
20 15
15 10
6 3 Scheduling Methods
Model Predictive Control Algorithms
Intelligent and Learning Algorithms
Optimal Algorithms
Decision Methods
Figure 9. The classification statistics of papers in theory.
Luo from the Chinese Academy of Science. Significant contributions were made from academia, indus try, and management agencies. For
Ap ri l 2022
more details on ICNSC, please visit the conference webpage at http:// icnsc2021.com/.
IEEE SYSTEMS, MAN, & CYBERNETICS MAGAZINE
49