Vol. 29, No. 2, JUNE 2022 IEEE Robotics & Automation Magazine

Citation preview

Displaying the world in your hands. Inventing new ways to interact. Force Dimension designs and manufactures the finest master haptic devices for leading-edge applications in research, medical, industry, and human exploration.

The ScanTrainer from Intelligent Ultrasound is an educational tool that uses real patient scans and curriculum-based teaching across obstetrics, gynecology, general medicine, and emergency medicine. The system integrated a customized delta.3 haptic device. Force Dimension Switzerland www.forcedimension.com [email protected]

Vol. 29, No. 2 JUNE 2022 ISSN 1070-9932 http://www.ieee-ras.org/publications/ram

FEATURES 10

Intelligent Blockchain-Enabled Communication and Services Solutions for Moving Internet of Things Devices By Ismaeel Al Ridhawi, Moayad Aloqaily, and Fakhri Karray

21

Proof of Work Is a Stigmergic Consensus Algorithm

33

Detect-Remove-Replace

46

The Franka Emika Robot

Unlocking Its Potential By Önder Gürcan

A Robotic Solution That Enables Unmanned Continuous 3D Printing By Hsieh-Yu Li, Yuchen Ma, M. Naufal A. Bin Miswadi, Long Nguyen Nguyen Luu, Liangjing Yang, Shaohui Foong, Gim Song Soh, Espen Sivertsen, and U-Xuan Tan

A Reference Platform for Robotics Research and Education By Sami Haddadin, Sven Parusel, Lars Johannsmeier, Saskia Golz, Simon Gabl, Florian Walch, Mohamadreza Sabaghian, Christoph Jähne, Lukas Hausperger, and Simon Haddadin

65

A RoboStack Tutorial

75

Learning Fast and Precise Pixel-to-Torque Control

85

Benchmarking Torque Control Strategies for a Torsion-Based Series Elastic Actuator

Using the Robot Operating System Alongside the Conda and Jupyter Data Science Ecosystems By Tobias Fischer, Wolf Vollprecht, Silvio Traversaro, Sean Yen, Carlos Herrero, and Michael Milford

A Platform for Reproducible Research of Learning on Hardware By Steffen Bleher, Steve Heim, and Sebastian Trimpe

By Barkan Ugurlu, Emre Sariyildiz, Ahmet Talha Kansizoglu, Erim Can Ozcinar, and Sinan Coruk

ON THE COVER This issue of IEEE Robotics and Automation Magazine features a special section of two papers on blockchain applications in robotics and automation.

©SHUTTERSTOCK.COM/BLUE PLANET STUDIO

If you like an article, click this icon to record your opinion. This capability is available for online Web browsers and offline PDF reading on a connected device. Digital Object Identifier 10.1109/MRA.2022.3166034

JUNE 2022

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

1

A Publication of the IEEE ROBOTICS AND AUTOMATION SOCIETY Vol. 29, No. 2 June 2022 ISSN 1070-9932 http://www.ieee-ras.org/publications/ram

EDITORIAL BOARD Editor-in-Chief Yi Guo ([email protected]) Stevens Institute of Technology (USA)

COLUMNS & DEPARTMENTS 4 FROM THE EDITOR’S DESK 6 PRESIDENT’S MESSAGE 97 SOCIETY NEWS 104 CALENDAR

Editors Elena De Momi Politecnico di Milano (Italy) Jindong Tan University of Tennessee (USA) Associate Editors Ming Cao University of Groningen (The Netherlands) Feifei Chen Jiao Tong University (China) Carlos A. Cifuentes Universidad del Rosario (Colombia) Kingsley Fregene Lockheed Martin (USA) Antonio Frisoli Scuola Superiore Sant’Anna (Italy) Jonathan Kelly University of Toronto (Canada) Ka-Wai Kwok The University of Hong Kong (Hong Kong) Xiang Li Tsinghua University (China) Perla Maiolino University of Cambridge (UK) Surya G. Nurzaman Monash University (Malaysia) Weihua Sheng Oklahoma State University (USA) Yue Wang Clemson University (USA) Enrica Zereik CNR-INM (Italy) Houxiang Zhang Norwegian University of Science and Technology (Norway) Past Editor-in-Chief Bram Vanderborght Vrije Universiteit Brussel (Belgium) RAM Column Manager Amy Reeder (USA) RAM Editorial Assistant Joyce Arnold (USA) COLUMNS Competitions: Hyungpil Moon (Korea) Education: Andreas Müller (Austria) From the Editor’s Desk: Yi Guo (USA) Industry News: Tamas Haidegger (Hungary)

2

•

Humanitarian Technology: Vacant Standards: Craig Schlenoff (USA) President’s Message: Frank Park (Korea) Regional Spotlight: Megan Emmons (USA) Student’s Corner: Francesco Missiroli (Germany) TC Spotlight: Yasuhisa Hirata (Japan) Women in Engineering: Karinne Ramirez Amaro (Sweden) IEEE RAS Vice-President of Publication Activities Todd Murphey (USA) RAM home page: http://www.ieee-ras.org/publications/ram IEEE Robotics and Automation Society Executive Office Peter Sobel Executive Director Amy Reeder Program Specialist [email protected] Advertising Sales Mark David Director, Business Development—Media & Advertising Tel: +1 732 465 6473 Fax: +1 732 981 1855 [email protected] IEEE Periodicals Magazines Department Kristin Falco LaFleur Journals Production Manager Patrick Kempf Senior Manager Journals Production Janet Dudar Senior Art Director Gail A. Schnitzer Associate Art Director Theresa L. Smith Production Coordinator Felicia Spagnoli Advertising Production Manager Peter M. Tuohy Production Director Kevin Lisankie Editorial Services Director Dawn M. Melley Staff Director, Publishing Operations IEEE-RAS Membership and Subscription Information: +1 800 678 IEEE (4333) Fax: +1 732 463 3657 http://www.ieee.org/membership_services/ membership/societies/ras.html

Digital Object Identifier 10.1109/MRA.2022.3166035

IEEE prohibits discrimination, harassment, and bullying. For more information, visit http://www.ieee.org/web/aboutus/whatis/policies/p9-26.html.

IEEE Robotics and Automation Magazine (ISSN 1070-9932) (IRAMEB) is published quarterly by the Institute of Electrical and Electronics Engineers, Inc. Headquarters: 3 Park Avenue, 17th Floor, New York, NY 10016-5997 USA, Telephone: +1 212 419 7900. Responsibility for the content rests upon the authors and not upon the IEEE, the Society or its members. IEEE Service Center (for orders, subscriptions, address changes): 445 Hoes Lane, P.O. Box 1331, Piscataway, NJ 08855 USA. Telephone: +1 732 981 0060. Individual copies: IEEE Members US$20.00 (first copy only), non-Members US$140 per copy. Subscription rates: Annual subscription rates included in IEEE Robotics and Automation Society member dues. Subscription rates available on request. Copyright and reprint permission: Abstracting is permitted with credit to the source. Libraries are permitted to

photocopy beyond the limits of U.S. Copyright law for the private use of patrons 1) those post-1977 articles that carry a code at the bottom of the first page, provided the per-copy fee indicated in the code is paid through the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923 USA; 2) pre-1978 articles without a fee. For other copying, reprint, or republication permission, write Copyrights and Permissions Department, IEEE Service Center, 445 Hoes Lane, Piscataway, NJ 08854. Copyright © 2022 by the Institute of Electrical and Electronics Engineers Inc. All rights reserved. Periodicals postage paid at New York and additional mailing offices. Postmaster: Send address changes to IEEE Robotics and Automation Magazine, IEEE, 445 Hoes Lane, Piscataway, NJ 08854 USA. Canadian GST #125634188 PRINTED IN THE U.S.A.

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

JUNE 2022

Meet the next medical pioneers �KUKA Innovation Award 2022 “The KUKA Robotics in Healthcare Challenge is a wonderful way to stimulate young scientists to create innovations for important societal relevant themes. For a number of years now, I have had the pleasure to be involved in the KUKA Innovation Award jury and every time I enjoy experiencing the enthusiasm and great ideas which are presented. It is an exceptional opportunity to bring research close to industry!” Researchers, developers and young entrepreneurs from all over the world submitted their concepts. Meet the selected finalists now at kuka.com/InnovationAward2022

www.kuka.com

Prof. Dr. Ir. Stefano Stramigioli, University of Twente Member of the international jury, Innovation Award 2022

JUNE 2022

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

3

FROM THE EDITOR’S DESK

New Ideas, Old Tricks By Yi Guo

A

s more and more countries ease coronavirus restrictions and open up, the global supply chain crisis does not seem to get better, and many industries are facing a shortage of labor. The old myth that robots will destroy jobs is proving to be far from the truth. In fact, the evidence With IEEE Robotics suggests that and Automation robots will be disruptive but Society financial ultimately benassistance approved eficial for labor markets [1]. In last year, we have a pandemicmade 10 popular ravaged world, it would be a RAM tutorial articles good thing to open access. have more robots to boost productivity affected by staff shortages and to look after older adults who are in need of help. Blockchain, a database that contains the history of whatever information it was designed to store, was once seen as the next big thing by the financial technology industry. However, it was recently discovered that building better blockchains is surprisingly hard [2]. To Digital Object Identifier 10.1109/MRA.2022.3165906 Date of current version: 13 June 2022

4

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

JUNE 2022

computer scientists, the ideas underlying blockchains are hardly new. For example, one of the main features to have publicly accessible blockchains to work is a “consensus mechanism.” The consensus mechanism has been studied by describing collective behaviors among schools of fish and flocks of birds since the 1950s and later widely used in designing the distributed cooperative control of mobile agents and autonomous robotic vehicles. My own research group has applied the distributed consensus principle in various applications of robotic and networked systems [3]. The ubiquity of networked systems makes the fundamental idea useful in many real-world engineering applications. This issue of IEEE Robotics & Automation Magazine (RAM) features a special section of two papers on blockchain applications in robotics and automation. I would like to thank the lead guest editor, Fabio Bonsignorio, who proposed the topic and managed the peer review process with fellow guest editors Sooyong Park, Eduardo Castello-Ferrer, Önder Gürcan, and Alexander Kapitonov. Although blockchain may not be popular in the robotics research community at this time, we hope that the special section raises

awareness and interest in multidisciplinary topics underlying our field. In addition to the special section articles, this issue contains tutorials and reproducible work. Tutorial articles are popular, evidenced by the large number of downloads and citations. With IEEE Robotics and Automation Society financial assistance approved last year, we have made 10 popular RAM tutorial articles open access (OA). In the next two years, we will make another 20 tutorials OA. Reproducible articles have an “R-Article” tag on them and are published as OA. A list of OA articles can be found at the RAM website. We hope this initiative helps disseminate innovation to a wider audience. Enjoy the issue! References

[1] “The world should welcome the rise of the robots. The Economist. https://www.economist.com/ leaders/2022/02/26/the-world-should-welcome -the-rise-of-the-robots (Accessed: Apr. 13, 2022). [2] “Is a greener, faster and more decentralised alternative to Bitcoin possible?” The Economist. https://www.economist.com/finance-and -economics/2022/01/01/is-a-greener-faster-and -more-decentralised-alternative-to-bitcoin-possible (Accessed: Apr. 13, 2022). [3] Y. Guo, Distributed Cooperative Control: Emerging Applications. Hoboken, NJ, USA: Wiley, 2017. 

RB-KAIROS+ COLLABORATIVE MOBILE MANIPULATOR

MOBILE SERVICE ROBOTICS

R

B

-V

O

G

U

I

R

B

-C

A

R

R

B

-R

O

B

O

U

T

RB-THERON

robotnik.eu

Robotnik provides AMR & mobile manipulators for industrial applications and R&D. All of them use ROS open architecture and include a number of advanced functions that simplify the d e v e l o p m e n t o f t u r n - ke y s e r v i c e r o b o t i c s applications.

PRESIDENT’S MESSAGE

Reaffirming Our Values as a Global Community By Frank Park

A

s I write this in mid-April, the latest hopeful news that the global pandemic might finally be ebbing has been swept aside by the horrific scenes of devastation and human suffering in Ukraine. Like other communities, roboticists throughout the world have also been deeply afI believe that we fected by this can be truly helpful t r a g e d y. I ’v e been part of vigwhen the time orous debates comes to rebuild online about the need to take colthe devastated lective action of educational some sort, from putting out a and research moral statement infrastructure. of conscience to offering financial and material support to those suffering from the war. Many have asked why IEEE has not put out a statement condemning the war. Here are some excerpts from IEEE’s recent position on this issue: The invasion of sovereign Ukraine by Russia is widely viewed as a violation of global norms, an assault on the conscience….IEEE’s relationship is with our members, not their countries, their employers, or institutions, and our apolitical, Digital Object Identifier 10.1109/MRA.2022.3169045 Date of current version: 13 June 2022

6

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

JUNE 2022

neutral stance on matters outside the organization’s technical scope allows us to engage with our members in every part of the world….Our core policies insist that we do not discriminate based on nationality; IEEE cannot expel some of our members due to the behavior of their governments, or limit access to IEEE journals and meetings based on nationality. I know that for some members who have called for stronger action— recently, Russia and Russians have been canceled on many fronts, from corporations to football teams to orchestras—this statement may not go far enough. Yet it must be acknowledged that figuring out where to draw the line is not always easy. It is legitimate to ask, for example, if cancelling those who bear no responsibility for the war, and may in fact themselves be victims, is justifiable, even if it is to achieve larger aims for the greater good. We can, however, reaffirm our values that are consistent with IEEE’s mission of advancing technology for humanity: the universal principles of respect for human dignity, seeking truth and fairness, behaving with integrity, and the open flow of information and the free exchange of ideas. We can begin by rejecting euphemisms and calling this a tragic, senseless, and unjustifiable war, a violent invasion of a peaceful sovereign neighbor.

History is replete with examples of people being dragged into conflicts by leaders who place their personal ambitions over the public good—and the public being powerless to stop them. Perhaps one big change is that we now live in a highly networked world in which moral outrage can quickly spread globally, and small groups of individuals have been empowered as never before. There are now many channels through which individuals can contribute to humanitarian relief efforts for those who have been displaced and victimized by the war. Aside from these individual efforts, what more can we do as a community? I believe that we can be truly helpful when the time comes to rebuild the devastated educational and research infrastructure. There are many steps we can take. For one thing, our existing programs on exchange, mentorship, and humanitarian technology can be easily expanded and adapted. Another immediate step is to increase our efforts and investment in education. There is general agreement that our educational efforts thus far have been too specialized with too much assumed knowledge—we have funded what people wanted to create rather than what people need, and sadly, much of the content has not aged well. In the coming months, I hope to tell you more about our efforts to significantly expand online education, with equitable and generous access for

and the ones inventing the future; many of us teach, and in fact, wrote the textbooks, and we have the funds to support our educators and researchers creating the content. I realize that these steps may seem small and trivial given the immediate existential struggles faced by those suffering from the war. I also know that for many, this war is not merely a regional conflict but something with profound implications for the global order. Our hearts go out to the Ukrainian people— to those in Ukraine and to those who find themselves refugees from the conflict—and to all others, in Russia and throughout the world, who are suffering from the consequences of this tragic and senseless war.

IEEE connects you to a universe of information!

IMAGE LICENSED BY INGRAM PUBLISHING

anyone in the world. The auOur hearts go out dience can be to the Ukrainian a n y one f rom industry to inpeople—to those s t itutions to academia, from in Ukraine and to those seeking those who find to quickly downthemselves refugees load a new skill to those wishing from the conflict. to learn the fundamentals of robotics as a part of a global robotics curriculum, similar to what the Association for Computing Machinery has developed for computer science. Our competitive advantages as a community are numerous: we are the most connected to robotics research

As the world’s largest professional association dedicated to advancing technological innovation and excellence for the benefit of humanity, the IEEE and its Members inspire a global community through its highly cited publications, conferences, technology standards, and professional and educational activities.

Visit www.ieee.org.

Publications / IEEE Xplore ® / Standards / Membership / Conferences / Education

ROBOTIC END - EFFECTORS

Measure all six components of force and torque in a compact, rugged sensor. Interface Structure

high-strength alloy provides IP60, IP65, and IP68 environmental protection as needed

Sensing Beams and Flexures

designed for high stiffness and overload protection protec ion withou without compromising resolutionn resolutio

High-Speed Electronics

interfaces for Ethernet, PROFINET, EtherNet/IP, Analog, USB, CAN EtherCAT, Wireless, and more

Silicon Strain Gages

provide high noise immunity, accuracy, and high factor-of-safety, standard on all F/T models

Engineered for high-performance and maximum stiffness, with the highest resolution and accuracy available, it’s the ultimate force/torque sensor. Only from ATI. www.ati-ia.com 919.772.0115

8

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

JUNE 2022

Since motors appeared in the early 19th century, they have been used in all types of electrical appliances and are now an indispensable part of our daily lives. Today, a huge number of motors are used in a wide range of applications, and it is claimed motors account for more than 55% of the world's power consumption. Therefore motor research is extremely important if we are to maintain our affluent lives while also perpetually conserving the global environment. We created these Nagamori Awards to bring vitality to technological research of motors and related fields, such as generators and actuators, and also to support the researchers and development engineers who strive each day to fulfill their dreams.

Nagamori Foundation decided six winners of the 8th Nagamori Awards, from whom the Grand Nagamori Award winner will be chosen on September 4th. The Grand Nagamori Award winner will receive 5 Million JPY and each Nagamori Award winner, 2 Million JPY.

The 8th Nagamori Awards Winners: Huijun Gao

Gianmario Pellegrino

Professor and Director, Research Institute of Intelligent Control and Systems, Harbin Institute of Technology

Full Professor, Department of Energy "Galileo Ferraris", Politecnico di Torino

Yunwei Ryan Li

Maryam Saeedifard

Professor and Acting Department Chair, Department of Electrical and Computer Engineering, University of Alberta

Associate Professor, School of Electrical and Computer Engineering, Georgia Institute of Technology

For contributions to the advanced control for mechatronic systems

For contribution to the PWM, control and converter topology of medium voltage high power industrial drives

Burak Ozpineci

Section Head, Vehicle and Mobility Systems Research Section, Building and Transportation Science Division, Oak Ridge National Laboratory

Low cost, high efficiency, compact electric motor drives for more electrified transportation systems Contact information: Nagamori Foundation

Address: 338 Kuzetonoshiro-cho, Minami-ku, Kyoto 601-8205, JAPAN Tel: +81-75-935-7731 E-mail: [email protected] https://www.nidec.com/en/nagamori-f/

Synchronous and PM-synchronous reluctance motor drives - theory, design, and control methods

For contributions to highly-efficient, power-dense and fault-tolerant multilevel converter-based medium-voltage drives

Akio Yamamoto

Professor, Graduate School of Frontier Sciences, The University of Tokyo

Pioneering research and development on theoretical models and applied systems for electrostatic film actuators

Intelligent BlockchainEnabled Communication and Services Solutions for Moving Internet of Things Devices By Ismaeel Al Ridhawi, Moayad Aloqaily, and Fakhri Karray

T

he monumental growth in the frequency and sophistication of connected moving objects has led to a plethora of solutions to enable continuous, seamless, and reliable connectivity and service delivery. Connected, autonomous, and artificial intelligence (AI)-enabled Internet of Things (IoT) moving devices are playing a significant role in the establishment of reliable wireless connectivity infrastructures for Beyond 5G Digital Object Identifier 10.1109/MRA.2022.3163081 Date of current version: 14 April 2022

10

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

JUNE 2022

(B5G) telecommunication networks. The advancements in ad hoc communication and moving platforms are now being recognized as alternative solutions toward connectivity in many complex environments for applications, such as autonomous transportation and robotics. Today’s availability of AI-enabled moving devices, incorporating advanced processing and communication capabilities, together with advanced decentralization-enabling technologies, such as intelligent blockchain, provide alternatives to autonomy for moving platforms in smart cities. In this article, we discuss recent advances in moving IoT devices, such as unmanned

1070-9932/22©2022IEEE

aerial vehicles (UAVs) and unmanned ground vehicles (UGVs), that adapt intelligent connectivity support, data collection, decision making, and blockchain technology to facilitate autonomous configurability and service provisioning for IoT networks. Background on 5G and 6G Networks The 5G architecture has laid out the foundation for a gradual movement from base station (BS)-centric to device-centric networks. Significantly smaller cell deployments using unused, high-frequency millimeter-wave bands ranging from 2.4 to 72 GHz are now used to support hundreds of times more data and capacity. Although 5G is still being deployed in many parts of the world, it still has great areas of improvement, especially with the aid of user-centric networks. IoT devices are no longer just service requesters but rather can be considered active participants in the 5G architecture. With their technological advancements, IoT devices can now participate in processing, storage, and communication. Such visions may not be realized in the current 5G variant but will most likely be seen in B5G deployments, such as 6G networks. Terahertz band communication envisioned for use in 6G networks will provide full support for real-time traffic demands for intelligent and excessively high data requirement applications for highly mobile IoT devices. 6G will go beyond traditional wireless cellular connectivity for mobile Internet usage and will support ubiquitous and seamless AI services for both core and IoT devices on the network. All IoT devices (e.g., fixed and mobile robots) must be willing to collaborate and cooperate using full and unrestricted secure connectivity to achieve the 6G vision, namely: 1) extremely fast downlink and uplink data rates of up to 10 Tbps, 2) extremely low latencies in the range of 10–100 microseconds, 3) increased network capacity of up to 1 Gb/s/m2, and 4) high energy efficiency for mobile resource-constrained devices. IoT devices have greatly advanced over the years, revolutionizing and reshaping their role in the technology industry. The IoT has now progressed to provide both fixed and mobile devices with advanced processing, storage, and communication capabilities as well as advanced machine learning (ML) and AI technologies. These intelligent fixed and moving IoT devices can range from small ordinary household appliances to sophisticated moving consumer, commercial, infrastructure, and industrial tools [1]. Vehicular and mobile IoT devices, such as UAVs and UGVs, have played a significant role in relaying data and will have a great impact in supporting B5G networks, such as 6G. Ad hoc

communication with the support of software-defined networking (SDN) and network function virtualization (NFV) has enabled moving robotic devices to communicate and share data without reliance on fixed telecommunication platforms [2]. Today, IoT devices with predictable mobility characteristics are serving as roadside units (RSUs) to enable connectivity to the core network for other moving IoT devices. Moving platforms have since evolved to include flying ad hoc networks (FANETs) to support ultrawide data delivery. Figure 1 depicts an illustrative view of the integration of intelligent fixed and moving IoT devices within next-generation networks. As is the case in many networking and service provisioning domains, the assurance of adaptive and continuous service and network availability and diversity cannot be achieved without reliance on autonomous, self-configurable, and self-manageable techniques. AI has played and continues to play a major role in system autonomy. Today, ML algorithms are providing high accuracy rates in decision making, thus maximizing reliability. To adapt to scalability and efficiency, federated learning (FL) was introduced as a decentralized ML mechanism to adhere to personal data integrity, security, and privacy. To adhere to the vision of 6G, autonomy and self-configurability should be adapted to supportive moving platforms with the aid of AI at both the edge and IoT devices. Plug-and-play AI (PnP-AI) was introduced to support the autonomous detection of the problem type and the features required [3]. The training data set, labels, evaluations, and models are chosen and executed without user intervention. It allows for the selection of ML algorithms based on the problem’s complexity in an effort to minimize runtime and maximize accuracy. Such a solution, if adapted at the network core and edge, will allow for optimal configurations to be adapted based on real-time network and device resource availability. Furthermore, to enable AI transparency, PnP-AI produces explainable models for both the problem domain and the intended consumers. Moving platforms can significantly benefit from PnP-AI, allowing for the dynamic reconfiguration of the virtualized resources and communication channels with the support of SDN and NFV. Achieving a cooperative behavior between moving IoT devices for capability and resource sharing purposes cannot be attained without the assurance of transaction authenticity and resource integrity. Relying on centralized solutions to authenticate and ensure secure transaction processing is lacking in many aspects, such as scalability, management, data manipulation, nonpersistency of data, single point of failure, and being prone to cyberattacks. Similarly, a distributed solution will also have drawbacks in terms of cost, network overhead, and complexity in terms of data integrity. Blockchain technology was introduced as an alternative solution to data management and transaction processing. It is a ledger-based system that enables decentralized management and maintains record copies of all transactions performed by the participants. These copies are securely stored at each participant’s JUNE 2022

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

11

side. This will ensure that data are immutable and can record new transactions based only on verification and consensus methods. Scalability, total integrity, data auditability, and transparency are all advantages of using blockchain. Integrating the advances in both AI and blockchain spawns a new direction referred to as intelligent blockchain, that provides promising potential to overcome challenges in service composition. Moving IoT devices will benefit from intelligent blockchain to allow devices to share and compose resources and services on demand in a secure decentralized transaction record-keeping manner. With that said, in this article, we provide potential solutions and directions for researchers that enable smart, connected, autonomous, and intelligent communication and networking infrastructures. Such solutions rely on the adaptation of the use of PnP-AI and FL in moving platforms to adhere to 6G connectivity requirements in terms of ultrafast connectivity and intelligent services. These connectivity requirements cannot be realized without the cooperation of robotic IoT devices (both on ground and in air) in terms of resource and service sharing. Moreover, to guarantee the authenticity and integrity of the decentralized cooperation process, we illustrate that intelligent blockchain-supported moving platforms are a promising solution.

●●

●●

●●

The contributions of this article are summarized next: We examine three aspects of moving platform management that need to be considered to achieve true autonomous smart moving IoT devices, namely: 1) intelligent connectivity, 2) intelligent resource management, and 3) intelligent service provisioning. We propose an intelligent blockchain-supported solution to enable an overall understanding of the services and capabilities of IoT devices in a moving network. We present performance evaluation results that compare the intelligent blockchain-supported solution against benchmark cooperative and noncooperative solutions to highlight the significance of the proposed scheme in terms of service availability, retrieval speed, and effectiveness against cybersecurity attacks.

Intelligent Connectivity Moving robotic devices rely on innovative communication technologies that use fixed and moving platforms to achieve consistent connectivity. Such devices require constant, consistent, and reliable connectivity to maintain adequate quality of service (QoS). This is not only necessary for traditional applications but is highly mandatory for tactile Internet applications (e.g., robotic platooning [4]).

Transportation

Governance

City Planning

Education

Infrastructure

Security

Healthcare Water/ Sanitation

Energy

Global Deep Learning Cloud

Fog Edge Node

Fog

Moving Base Station Unmanned Ground Vehicle

Edge Node

UAV Charging Station Connected IoT Devices

Moving IoT Devices

Federated Learning

RSU

Figure 1. An illustrative view of a smart environment supported by connected intelligent fixed and moving devices. RSU: roadside unit.

12

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

JUNE 2022

Current Trends in Intelligent Connectivity Mobile Robot Swarm Communication Communication between mobile robots in swarm scenarios plays a significant role in many real-world scenarios, such as search and rescue, battlefield communication, and UAV swarm. Such robots move in groups in accordance with group mobility models. In [5], an energy-efficient cooperative relay strategy is used for mobile robot networks. The solution uses a clustering strategy in which each robotic device computes a relative energy rank or residual energy in the cluster used to select the cluster head. Moreover, relay selection and caching strategies are used for inter- and intracluster communication. Such cluster techniques and the use of mobility models play a major role in swarm communication and movement. Vehicular Ad Hoc Networks The high cost incurred from RSU deployment and management for vehicular networks has prevented them from being widely used in practice despite their importance. The usage of buses as mobile RSUs (mRSUs) has been shown to be more beneficial in terms of deployment cost and usability. In [6], the authors investigated the usage of mRSUs through real-world measurements, experiments, analysis, and simulations. The results have shown that the same levels of throughput, contact, and intercontact time can be maintained as in static RSUs, depending on the deployment environment. FANETs The use of UAVs, whether in a fixed location or moving, has been adapted in numerous studies to support both vehicular and mobile device connectivity. UAVs supported by FANETs can not only provide communication-level capabilities to uplift severe connectivity intermittence and delayed data delivery for fixed infrastructure users but can also provide assistance for vehicular networks. For instance, in [7], the authors considered UAVs to serve as intermediate relays between disconnected vehicles and the infrastructure. UAVs collect and carry data to RSUs that are not within the communication range of the disconnected vehicles. The work conducted experiments to determine the number of UAVs required to completely heal a disrupted vehicular ad hoc network (VANET). UAV-Supported Core Network The use of flying platforms has been considered in research to enable connectivity for moving devices, either directly or as flying relay BSs to fixed BSs connected to the core network. In [8], the authors consider the synergy of FL and blockchain to enable drone edge intelligence. This enables UAVs to form communication links over smart environments to maintain connectivity, enhance network performance, attain load balancing, and manage energy consumption. With the UAVs’ FL capabilities, the devices can participate in model training, which will significantly speed up the training process. With

blockchain, a global model is attained from the cooperating intelligent drones and downloaded to the moving devices on the go. Proposed Solutions Today, research in the field of advanced ML aims at applying intelligent learning to mobile networks. This will ensure that optimal levels of self-configurability are achieved at different communication levels, namely: 1) between moving robotic IoT devices using VANETs and mobile ad hoc networks (MANETs) to attain highly accurate movement of robotic swarms and achieve extended applicability to a multitude of scenarios 2) between moving robotic IoT devices and ground BSs for extended MANET/VANET connectivity to achieve high levels of communication reliability 3) between moving robotic IoT devices and aerial BSs using AI-supported collaborative MANET/VANET and FANET communication and networking 4) between the FANET and the core network to ensure highly reliable aerial backhaul communication. Given that most IoT devices (i.e., ground, robotic, mobile, vehicular, or aerial) have significant processing, storage, and communication capabilities, the adaptation of ML solutions at different communication layers with the support of FL, PnP-AI, and intelligent blockchain will ensure accurate and reliable self-configuration that attains to strict QoS and energy requirements. Moving IoT devices, such as in robotic swarm scenarios, cannot attain reliable levels of connectivity using traditional communication that relies on fixed BSs. As such, intelligent collaborative ad hoc communication will provide high levels of support for IoT and robotic device connectivity, which may lead to the replacement of the core network. Furthermore, not only are communication and networking services made available through such intelligent device-to-device connectivity, but also, intelligent communication with flying aerial devices can be used to achieve high levels of connectivity and attain the needed services. Cooperative MANET/ VANET and FANET communication will achieve optimal levels of end-to-end service delivery and provide a smooth, soft handover between the devices while moving. Table 1 provides an overview of the services that will receive benefits from the adaptation of a cooperative ground/aerial communication architecture. Future Directions in Intelligent Connectivity Forthcoming communication solutions for IoT and robotic devices must rely on intelligent communication that establishes end-to-end communication spectrums based on a deviceby-device necessity basis. End-to-end communication is achieved through an overlay layer that relies on IoT devices and infrastructure resource sharing [9]. Overlays may be created for FANETs, VANETs, or all cooperative core and ad hoc networks to provide optimal data routing paths in terms of reduced data delivery latency and enhanced QoS according to JUNE 2022

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

13

service-specific requests. Incentivized mechanisms may be adapted to ensure the availability of the needed communication resources and the trustworthiness of the participants. The overlay layer is supported by AI-enabled IoT devices that effectively analyze and predict the mobility characteristics and location predictability of the IoT and robotic devices. In essence, this will lead to the reliability of continuous communication availability through moving and fixed BSs, ensuring that they are available at destination locations for IoT and robotic devices. A relative velocity mechanism can be adapted to neighboring IoT devices to determine their movement characteristics [10]. This will, in essence, aid the prediction process and support continuous service availability for clusters. Given that aerial IoT devices such as UAVs are limited in power, such devices must be flown at optimized heights, durations, and distances to prolong their availability for communication services. UAV charging stations can be supported by ground robotic devices available near ground BSs and RSUs to speed up the charging process. Such charging stations may be either fixed or mobile and may be involved in energy trading [11]. Travel path planning for UAVs is configured in accordance with mobility and power consumption models with the support of different learning strategies that consider features such as flight height, velocity, and different mobility characteristics. Moreover, device cluster formation solutions can be considered to ensure prolonged communication connectivity with other nearby UAVs, RSUs, and BSs. To achieve such a high level of self-configurability, a distributed and decentralized learning strategy must be considered for fast and on-demand decision making. Challenges Having IoT devices participate in the process of communication and service delivery may raise a number of issues in terms of trust and reliability. IoT robotic devices, whether

aerial or ground, are limited in power resources, and yet they require significant power usage for navigation and movement. For instance, UAVs have different flying and movement characteristics, where moving at faster speeds results in an accelerated depletion of battery power. With that said, for such IoT devices to participate in the communication process and serve as mobile BSs, the power needed to receive, store, process (whether intelligently using ML or traditionally), and transmit data would accelerate the battery power depletion process. Intelligent optimization techniques must be considered to ensure that proper communication coverage and power usage are incorporated for the participating IoT devices. Examples of some challenges that may arise in terms of communication are coverage overlapping, device movement, and travel optimization. Cooperation between the participating IoT devices and other moving platforms, such as vehicles, can play a significant role in ensuring continuous device connectivity and task offloading. The optimization of charging station placement and movement could provide support to robotic device service continuity. Connectivity cost is a challenging factor and requires selfconfiguring solutions to determine alternative cost-effective methods for communication. Given that participants in the communication process are distributed and mobile, the usage of new federated and distributed learning solutions at the edge and on IoT devices, as well as integrating such solutions together with centralized mechanisms such as global deep learning, can provide support to device connectivity and resource management. Given that both fully centralized and decentralized solutions cannot achieve optimal performance results, collaborative mechanisms that utilize cooperative mobile and fixed IoT devices, in addition to fog and cloud entities, are considered contemporary solutions with a plethora of challenges to be discovered and solved for by researchers. Security and trust are two major contributors to the success of decentralized solutions. The involvement of IoT

Table 1. The type of services that are made available with the adaptation of a UAV-supported framework and some qualitative analysis. Type of UAV Service Role of UAV

Enhancement Features

Application

Benefits

Blockchain Usage Potential

UAV-assisted Aerial small Network cell, BS, and backhaul

Extend network stability, capacity, reliability, and coverage

Location/proximity-based services and high-mobility device-to-device service (e.g., road and traffic conditions, content sharing, and autonomous driving)

Continuous network availability and end-to-end IoT device connectivity

Participation of UAVs granted autonomously through a consensus method

UAV-assisted Aerial small cell, RAN BS, backhaul, and spectrum control

Extend GenericRAN coverage and capacity, portable data offload

Delay-tolerant 5G services, power Reduced bandwidth, reduction through data distribution load balance, and and augmentation, and ground reduced latency sensor data collection

Faster decentralized authentication of transactions for FANETs

On-demand UAV cloud/edge-based processing, distributed vehicle model training, FL, content caching, and multimedia services

Maintain authenticity and integrity of participant UAVs and their resources/ services offered

UAV-assisted Distributed Extend cloud cloud/fog cloud and edge service coverage units, virtual BS and capacity

RAN: radio access network.

14

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

JUNE 2022

Service availability, reduced latency, data security, and privacy

devices in the communication and service delivery processes must not compromise system and device security. As such, solutions must ensure that participants are authorized to participate in the communication process. This can be achieved only through decentralized authentication mechanisms. Blockchain technology is a decentralized transaction processing solution that can be adapted for use by IoT and robotic devices. Any service delivered through a number of participating IoT devices can be authenticated through the blockchain to ensure that no manipulation of the original service occurs, and if there is some sort of service manipulation (e.g., service composition), this manipulation is authenticated and secure. Some of the challenges that may arise are in terms of intelligent blockchain adaptability and efficiency. Intelligent Resource Management As more IoT and robotic devices share resources and capabilities (i.e., processing, storage, and communication), resource management becomes an intricate problem. Economic and performance factors are important when determining the usage of device and network resources. Mobile robotic and IoT devices, such as UAVs and UGVs, have dynamically changing resources and thus require constant resource monitoring, utilization, configuration, allocation, and scheduling. With AIsupported resource management that relies on advanced ML solutions, such as deep reinforcement learning (DRL), resource utilization can be achieved both in a centralized fashion (e.g., cloud) and in a distributed fashion (e.g., fog). Current Trends in Intelligent Resource Management Cooperative resource management solutions have been considered in earlier works. For instance, in [12], an intelligent resource management approach is considered that uses connected UAVs to collect virtual reality content for wireless transmission to users. Specific visible content is extracted from the 360° virtual reality data and is sent to users to maintain a reduction in the traffic load over backhaul and radio access links. Small BSs are used to cache popular virtual reality data extracted content. The content caching and transmission mechanisms are formulated as an optimization problem whose goal is to ensure that the content transmission delay meets the virtual reality delay target for each user. Distributed deep learning is used, which combines liquid state machines with echo state networks to determine optimal caching and transmission strategies. Furthermore, in [13], a multidimensional resource management solution is developed for UAV-assisted vehicular networks. The solution assumes that both macro eNodeB BSs and UAVs are equipped with mobile edge computing (MEC) servers and cooperatively make connectivity and resource allocation decisions for vehicular nodes. A distributed optimization problem is considered to maximize the number of offloaded tasks while adhering to stringent QoS requirements. Given that the offloaded tasks are delay sensitive, the problem is solved by using reinforcement learning. A

multiagent, deep deterministic policy gradient-based algorithm is used to rapidly make vehicle association and resource allocation decisions, where each MEC server acts as a learning agent. Future Directions in Intelligent Resource Management With the availability of diversified and resource-rich IoT and robotic devices, distributed and decentralized AI-supported cooperative strategies that rely on blockchain technology have become more effective. For instance, UAVs can act as cluster heads for moving IoT and robotic devices. Those devices that have similar mobility characteristics can be clustered together, and the UAV can act as either a relay or a service node. Transactions processed in the form of resources, services, communication, and other capabilities can be authenticated using a private blockchain for the cluster. Blocks can be created to record the services shared and added to the blockchain. The UAV can adapt advanced resource management solutions to determine whether resources are available for sharing within the same cluster or neighboring clusters. When the service request cannot be processed by the servicing UAV, the request can be forwarded to the nearest edge device. The edge device can apply the necessary resource scheduling algorithms in accordance with the requests and currently available resources to meet current and future resource demands. A public blockchain can be used for intracluster cooperation to record services added to the blockchain. Additionally, online and offline scheduling algorithms should also be considered. Furthermore, the use of DRL is one way to ensure that complex resource demands are met by applying new configuration policies continuously to meet QoS and quality of experience requirements, resulting in reliable and continuous service availability. Intelligent Blockchain-Supported Service Provisioning Given that most service requests are tailored (i.e., device or user specific), the provisioning of such services through centralized mechanisms is infeasible. For instance, assume that a robotic IoT device is requesting a service that is distinct from the device’s specifications and user preferences. Centralized cloud-based solutions may not have the requested service readily available, and the processing of such a request may lead to intolerable latencies and service delivery costs. In essence, the reliance on service composition solutions that integrate the capabilities and resources of edge and IoT devices is considered an alternative to ensure that service requirements are met. Moreover, this will enable service trading to occur at the edge between IoT devices. Service composition is the process of using the capabilities of different cooperating edge and IoT devices to compose a device-specific service. To do so, the process begins with resource and capability advertising followed by service clustering for faster and more accurate service lookup. As such, a service is composed of a JUNE 2022

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

15

e

r5

ste

Clu Fog

Road Side Unit (RSU) Federated Learning

UAV Charging Station

RSU1

Fixed Base Station (FBS)

r2

ste

Clu

Service Task Device 3

Edge Node

ce

rvi

Se

Radius Elevation Angle

Coverage

Height

Private Blockchain

MBS1 Cluster

Service Cluster 4

Service Chord1

Service Cluster 1

Service Task Device 2

Global Deep Learning

Moving Base Station (MBS)

Public Blockchain

Fog

r5

ste

Clu

RSU2

Edge Node

ce

rvi

Se

C

lu

e st

r6

Service Task Device 4

ce

i

rv

Se

Relay Node 3 Service Task Device 5

Private Blockchain

Control Flow

Unmanned Ground Vehicle

Service Task Device n

Se

Data Flow

MBS2 Cluster

Service Cluster 4

Relay Node 5

Service Chord2

Service Cluster 1

Relay Node 4

rvi

lu s ter

ce C

2

Figure 2. An overview of a blockchain-enabled system that relies on edge device service capabilities to compose services and provide communication connectivity for IoT and robotic devices.

ce

rvi

r

Se

Se

c vi

lu

C

e st

r6

Service Task Device 1

Relay Node 2

Se

Relay Node 1

Communication and Service-Specific Overlay

c rvi

e C lus ter 3

JUNE 2022

c

•

rvi

IEEE ROBOTICS & AUTOMATION MAGAZINE

Se

• e C lus ter 3

16

lookup and available subservices provided by the edge and IoT devices. Named-defined networking can play a significant role in accurate service retrieval, given that it is a contentcentric architecture for the future Internet [14]. Current Trends in Intelligent Blockchain-Supported Service Provisioning Service composition is highly supported with the use of intelligent service agents (which reside within the IoT devices) to aid in the discovery, advertising, and communication processes of available services with other devices. Clustered servicespecific overlays are formed for each composite service request. Such overlays are formed from multilayered service clusters. Each layer consists of a diversified set of service clusters, such that each cluster withholds particular service capabilities and resources. Devices within the clusters have direct ad hoc communication with other devices (i.e., inter- and intracluster communication) or are supported by UAVs for communication, hence forming a chord [15]. As such, the process of service discovery, both from within a service cluster and outside of it, is accelerated with the aid of UAVs. Service clusters are formed on the basis of service capability comparisons, where variations in service semantics and functionality syntax are compared to determine service similarity [9]. Furthermore, multichords are formed, where each service cluster in a chord acts as a service discovery point to other service clusters from a different chord using direct or UAV-supported communication. Figure 2 depicts an overview of such a proposed solution that provides an overall knowledge of the available services and capabilities of the IoT devices in the network. The process of service composition through resource and service sharing using service-specific overlays must be authenticated for trustworthiness and security. As such, a trustworthy and secure transaction processing solution is highly essential. Blockchain technology supports secure transaction processing in a decentralized fashion. In [16], an on-demand drone deployment solution for content caching is being considered. A neural-blockchain approach has been developed to support the drone caching strategy to increase moving network reliability and network flattening. Given that the service composition process proposed in this work is decentralized, the blockchain is highly favorable. Service composition among trusted entities can be achieved through a private blockchain, whereas service composition that involves IoT and robotic devices can adapt to the use of public blockchain. Such a blockchain would represent the set of services offered by IoT devices needed to compose a service. In essence, the formed service composition path can be applied to the blockchain. Blockchain technology will play a significant role in the decentralized management of collaborative multirobots not only for data accessibility but also to share representation, goals, and trust [17]. UAVs and UGVs can incorporate the use of blockchain to maintain consistent service for IoT device clusters. For instance, UGVs may be deployed in traffic

intersections to serve as mobile charging stations for UAVs. As IoT devices move in a cluster on the road and are being serviced by UAVs for continuous connectivity and service delivery, those devices can ensure that their battery power is not depleted by using the blockchain secure state, resource, and capability exchange of the collaborating UAVs and UGVs. Such a similar approach was presented in [18], where multidrones adapted a decentralized blockchain-enabled management solution to provide continuous monitoring and detection of urban areas during COVID-19 to maintain social distancing, sensitization, and medical supply delivery while avoiding drone collision with one another. Future Directions in Intelligent BlockchainSupported Service Provisioning With the aid of intelligent blockchain agents, each IoT device using the blockchain generates a unique address for each service composition participation to maintain anonymity to nonparticipants. This would also allow for transaction tracing and recordkeeping to take place. In essence, a blockchain is generated by each IoT device within a cluster and can be formed either within the same cluster or over multiple clusters, depending on the type of composed service. Figure 3 illustrates the process of an intelligent blockchain formation. The blockchains (B 1, B 2, f, B n) are formed uniquely for each service composition process (S 1, S 2, f, S n). Each block B n is created with the aid of IoT device blockchain agents that generate a unique address for each service participation. Assuming that all participant IoT devices are within the same cluster, a private blockchain is formed. The service composition path is chosen in accordance with the similarity of the service capabilities provided by the IoT devices (Service simcomp) in terms of their descriptive properties [19]. With that said, the enhanced service provided by a device (Out S n) should be compared against the capabilities of the next device in the composition process (In S n) and that of what is expected to be the output in terms of the service enhancements of the device in the next time step (Out Sn + 1). The process of blockchain formation is repeated for every successful service composition. Additionally, given that the participating IoT devices have self-learning and self-configuring capabilities with the support of PnP-AI, the FL used in the service provisioning process can also be supported by blockchain. Since FL adapts local training using the devices’ local data to attain local models, the models can be verified with the blockchain through consensus before being sent to the edge or cloud. As such, the adaptation of blockchain and PnP-AI solutions in cooperative networks will nevertheless advance both connectivity and service provisioning. Performance Evaluation: A Scenario Progression toward intelligent cooperative solutions has been shown to be beneficial to both service requesters and providers. To test the effectiveness of the proposed model, we have JUNE 2022

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

17

extended our earlier analysis in [20] and conducted simulations using Network Simulator 3 (NS-3). Simulation settings are summarized in Table 2. All moving devices (UAVs and IoT devices) participate in the service/resource sharing and trading process in the proposed AI-enabled cooperative UAV and traditional cooperative UAV schemes. The 5G and Wi-Fi wireless communication protocols are adapted for device-toBS and device-to-device communication, respectively. The proposed AI-enabled cooperative solution enables UAVs to move toward highly dense IoT device locations to process alltime connectivity and a faster service hit ratio. The results attained have provided evidence that adapting an AI-enabled blockchain-supported cooperative UAV solution significantly enhances both connectivity and service delivery significantly, improving a FANET’s service provisioning and management process in addition to its lifetime.

Tests were conducted to evaluate UAV power usage over time when UAVs act as aerial BSs. By enabling UAVs to cooperate and predict when and how many BSs are needed to provide coverage for a set of moving IoT devices, the total residual energy is maintained over a longer period, as shown in Figure 4. This will, in essence, lead to higher service availability in the form of service hit ratio and packet delivery. Figure 5 outlines the delivery success ratio as the number of devices grows in the serviced area supported by intelligent UAVs. It is clear that as more devices are available, the AIenabled cooperative solution will ensure that high delivery success rates are maintained. Figure 6 depicts the number of requests that have been successfully delivered to the requester. The request may be composed of a number of service tasks. The results show that traditional methods cannot maintain a high service hit ratio as the number of simultaneous requests

Fog

Edge Node Private Blockchain B2

B1

Bn

Composite Service Servicesim_comp(OutS1, OutSn) Servicesim_comp(OutS1, OutS2) OutS1

S1

InS2

S2

OutS2

InSn

Sn

Servicesim_comp(OutS1, InS 2) Service Cluster 1

Se

ce

Clu

ste

r2

Ser

vice

Se

rvi

ce C

lus ter

3

Se

rv ice

Cl

us

te

r6

rvi

Clu

ster

5

Service Cluster 4 Figure 3. An illustration of an intelligent private blockchain created from different service capabilities used to compose and deliver device-specific services.

18

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

JUNE 2022

Table 2. Simulator Settings. Numerical Values

Communication protocol

IEEE 802.11ax (for device to device and device to RSU) gNodeB with 1-Gbps bandwidth (for device to BS)

Area

1,500 m × 1,500 m

Number of fixed BSs

1

Number of RSUs

5

Number of UAVs

20–50

Number of IoT devices

100–1,000 moving devices

UAV placement

Random: uniform distribution (initially)

Mobility model

Random: waypoint (for moving devices)

UAV flight height

50 m

Transmission power

0.1 W (for UAVs) 0.01 W (for moving devices)

Idle power

0.02 W (for UAVs) 0.001 W (for moving devices)

Blockchain hash algorithm

SHA-256

80 70 60 50 40 30 20 10 0

100 95 90 85 80 75 70 65 60 55 50

AI-Enabled Cooperative UAVs Cooperative UAVs Non-Cooperative UAVs 100 200 300 400 500 600 700 800 900 1,000 IoT Devices

Service Hit Ratio

Figure 5. The packet delivery success rate for service requests as the size of the network grows.

100 90 80 70 60 50 40 30 20

AI-Enabled Cooperative UAVs Cooperative UAVs Non-Cooperative UAVs 5

10 20 30 40 50 60 70 80 90 100 Service Requests

Figure 6. The retrieval success rate of composite services for simultaneous requests.

1

2

3

4

5 6 7 Time Units

8

9

10

AI-Enabled Cooperative UAVs Cooperative UAVs Non-Cooperative UAVs Figure 4. Comparing UAV power usage to deliver service requests.

11

Malicious Services (%)

Residual Energy (%)

Simulation Parameters

Lessons Learned and Concluding Remarks Adapting a PnP-AI solution at the edge and IoT devices to support an autonomous moving platform framework is indeed beneficial but complex and challenging at the same time. Two layers of learning are considered, namely, one that targets the problem domain (e.g., the type of device-specific service requested) and another that targets the management aspect of ML (e.g., training, learning, and validation). Since this might be resource consuming, the management layer must be distributed and decentralized. Hence, both FL and

Packet Delivery Success Rate (%)

increases. As such, service/capability discovery and retrieval with the aid of different ML algorithms are needed to maintain continuous and high hit ratios. Figure 7 outlines the importance of the use of blockchain in a cooperative environment. Devices share their resources and services, but some services might be malicious. The blockchain-enabled solution maintains a near-zero rate of malicious services shared. On the contrary, centralized and cooperative solutions that do not rely on malicious service detection schemes or those with limited capabilities incur low detection rates of malicious cyberattacks and lead to higher rates of malicious service availability.

30 25 20

Blockchain-Enabled Cooperative Centralized

15 10 5 0

100 200 300 400 500 600 700 800 900 1,000 Services Shared

Figure 7. The percentage of malicious services shared between IoT devices.

JUNE 2022

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

19

blockchain should be adapted not only to support the service provisioning and connectivity aspect of moving IoT devices but also to manage the learning mechanism that is adapted at the edge and end-devices. Today, all IoT and robotic devices are connected, and a significant number are mobile. With that said, the use of intelligent ad hoc communication along with other forms of moving platforms is instrumental. Moving platforms equipped with AI solutions can provide enhanced and more efficient device connectivity and service provisioning. It can be concluded that to achieve true self-configurability, the following must be considered: 1) intelligent connectivity, 2) resource management, and 3) service provisioning. Furthermore, the use of PnP-AI solutions and blockchain are major factors that enable true self-configurability for the upcoming next-generation networks, namely, 6G. This article examined those three aspects as well as introducing an intelligent blockchain-supported solution to enable an overall knowledge of the services and capabilities of IoT devices in a moving network. The performance evaluation results highlighted the significance of the proposed scheme in terms of service availability, retrieval speed, and effectiveness against cybersecurity attacks. References [1] X. Huang, X. Yang, Q. Chen, and J. Zhang, “Task offloading optimization for UAV-assisted fog-enabled Internet of Things networks,” IEEE Internet Things J., vol. 9, no. 2, pp. 1082–1094, 2022, doi: 10.1109/ JIOT.2021.3078904. [2] A. Papa, R. Durner, L. Goratti, T. Rasheed, and W. Kellerer, “Controlling next-generation software-defined rans,” IEEE Commun. Mag., vol. 58, no. 7, pp. 58–64, 2020, doi: 10.1109/MCOM.001.1900732. [3] I. A. Ridhawi, S. Otoum, M. Aloqaily, and A. Boukerche, “Generalizing AI: Challenges and opportunities for plug and play AI solutions,” IEEE Netw., vol. 35, no. 1, pp. 372–379, 2021, doi: 10.1109/ MNET.011.2000371. [4] D. St-Onge et al., “Planetary exploration with robot teams: Implementing higher autonomy with swarm intelligence,” IEEE Robot. Autom. Mag., vol. 27, no. 2, pp. 159–168, 2020, doi: 10.1109/MRA.2019.2940413. [5] Y. Zhang, T. Liu, H. Zhang, and Y. Liu, “LEACH-R: LEACH relay with cache strategy for mobile robot swarms,” IEEE Wireless Commun. Lett., vol. 10, no. 2, pp. 406–410, 2021, doi: 10.1109/LWC.2020.3033039. [6] J. Heo, B. Kang, J. M. Yang, J. Paek, and S. Bahk, “Performance-cost tradeoff of using mobile roadside units for V2X communication,” IEEE Trans. Veh. Technol., vol. 68, no. 9, pp. 9049–9059, 2019, doi: 10.1109/ TVT.2019.2925849. [7] M. Khabbaz, J. Antoun, and C. Assi, “Modeling and performance analysis of UAV-assisted vehicular networks,” IEEE Trans. Veh. Technol., vol. 68, no. 9, pp. 8384–8396, 2019, doi: 10.1109/TVT.2019.2911986. [8] S. H. Alsamhi et al., “Drones’ edge intelligence over smart environments in B5G: Blockchain and federated learning synergy,” IEEE Trans. Green Commun. Netw., vol. 6, no. 1, pp. 295–312, 2022, doi: 10.1109/TGCN.2021.3132561. [9] M. Aloqaily, I. A. Ridhawi, H. B. Salameh, and Y. Jararweh, “Data and service management in densely crowded environments: Challenges, opportunities, and recent developments,” IEEE Commun. Mag., vol. 57, no. 4, pp. 81–87, 2019, doi: 10.1109/MCOM.2019.1800624.

20

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

JUNE 2022

[10] I. Al Ridhawi, M. Aloqaily, B. Kantarci, Y. Jararweh, and H. T. Mouftah, “A continuous diversified vehicular cloud service availability framework for smart cities,” Comput. Netw., vol. 145, pp. 207–218, Nov. 2018, doi: 10.1016/j.comnet.2018.08.023. [11] M. Aloqaily, A. Boukerche, O. Bouachir, F. Khalid, and S. Jangsher, “An energy trade framework using smart contracts: Overview and challenges,” IEEE Netw., vol. 34, no. 4, pp. 119–125, 2020, doi: 10.1109/ MNET.011.1900573. [12] M. Chen, W. Saad, and C. Yin, “Echo-liquid state deep learning for 360° content transmission and caching in wireless VR networks with cellular-connected UAVs,” IEEE Trans. Commun., vol. 67, no. 9, pp. 6386–6400, 2019, doi: 10.1109/TCOMM.2019.2917440. [13] H. Peng and X. Shen, “Multi-agent reinforcement learning based resource management in MEC-and UAV-assisted vehicular networks,” IEEE J. Sel. Areas Commun., vol. 39, no. 1, pp. 131–141, 2021, doi: 10.1109/ JSAC.2020.3036962. [14] O. Karrakchou, N. Samaan, and A. Karmouch, “ENDN: An enhanced NDN architecture with a p4-programmabie data plane,” in Proc. 7th ACM Conf. Inf.-Centric Netw., New York, NY, USA: Association for Computing Machinery, 2020, p. 1–11, doi: 10.1145/3405656.3418720. [15] X. Meng and D. Liu, “GeTrust: A guarantee-based trust model in chord-based P2P networks,” IEEE Trans. Dependable Secure Comput., vol. 15, no. 1, pp. 54–68, 2018, doi: 10.1109/TDSC.2016.2530720. [16] V. Sharma, I. You, D. N. K. Jayakody, D. G. Reina, and K. R. Choo, “Neural-blockchain-based ultrareliable caching for edge-enabled UAV networks,” IEEE Trans. Ind. Informat., vol. 15, no. 10, pp. 5723–5736, 2019, doi: 10.1109/TII.2019.2922039. [17] S. H. Alsamhi and B. Lee, “Blockchain-empowered multi-robot collaboration to fight COVID-19 and future pandemics,” IEEE Access, vol. 9, pp. 44,173–44,197, 2021, doi: 10.1109/ACCESS.2020.3032450. [18] S. H. Alsamhi, B. Lee, M. Guizani, N. Kumar, Y. Qiao, and X. Liu, “Blockchain for decentralized multi-drone to combat COVID-19 and future pandemics: Framework and proposed solutions,” Trans. Emerg. Telecommun. Technol., vol. 32, no. 9, p. e4255, 2021, doi: 10.1002/ ett.4255. [19] I. Al Ridhawi, Y. Kotb, and Y. A. Ridhawi, “Workflow-net based service composition using mobile edge nodes,” IEEE Access, vol. 5, pp. 23,719–23,735, Oct. 2017, doi: 10.1109/ACCESS.2017.2766068. [20] M. Aloqaily, O. Bouachir, A. Boukerche, and I. A. Ridhawi, “Design guidelines for blockchain-assisted 5G-UAV networks,” IEEE Netw., vol. 35, no. 1, pp. 64–71, 2021, doi: 10.1109/MNET.011.2000170.

Ismaeel Al Ridhawi, Kuwait College of Science and Technology, Doha District, Kuwait. Email: [email protected]. Moayad Aloqaily, Machine Learning Department, Mohamed Bin Zayed University of Artificial Intelligence, Masdar City, United Arab Emirates. Email: moayad.aloqaily@ mbzuai.ac.ae. Fakhri Karray, Machine Learning Department, Mohamed Bin Zayed University of Artificial Intelligence, Masdar City, United Arab Emirates, and University of Waterloo, Canada. Email: [email protected] and [email protected]. 

©SHUTTERSTOCK.COM/ELENABSL

Proof of Work Is a Stigmergic Consensus Algorithm Unlocking Its Potential By Önder Gürcan

T

he proof-of-work (PoW) algorithm has been initially used in Bitcoin and is now one of the mainstream consensus algorithms to create immutable ledgers of transactions. However, it is still not very well understood, and as such, its entire strength remains undervalued. While some researchers classify it as a Nakamoto consensus algorithm, some other researchers do not even consider it as a consensus algorithm. In this article, the objective is to shed light on classifying PoW. To this end, we make a theoretical analysis from a coordination perspective, and then we conclude that PoW is a

Digital Object Identifier 10.1109/MRA.2022.3165745 Date of current version: 22 April 2022

1070-9932/22©2022IEEE

stigmergic consensus algorithm where the trace left by an action in the blockchain through indirect coordination of agents stimulates a subsequent action and eventually creates a single chain of blocks. In other words, PoW is a form of stigmergic coordination toward emergent behavior of a blockchain system. Based on this identification, we further analyze the fundamental properties of PoW. Finally, we reengineer PoW by improving its missing stigmergic mechanisms, and we provide a discussion. Background on PoW Since its launch in late 2008 [1], Bitcoin has experienced a steady growth in terms of involvement, transaction volume, and market capitalization (https://www.coinlore.com/all_coins) JUNE 2022

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

21

and remains the dominant blockchain system in the market. This success is achieved thanks to the use of technologies that already existed for building a decentralized, public, transparent, immutable, trusted, and ordered log of transactions known as blockchain. A blockchain system enables its participants to collectively create a distributed economic, social, secure, and technological framework in which everyone can join (or quit) and In PoW, to reduce the transact between two without needing to trust number of alternative each other to have a trusted third party and blocks significantly, have a global view. Bitcoin employs the the agents are forced PoW algorithm [2] for maintaining such an to solve a very hard agreement in a trustless network by including computational puzzle. mechanisms that ensure that the effort of block creation is represented within the block submitted by its creator. Technically speaking, in Bitcoin and all other PoW blockchain systems, there are two main kinds of participants: users and miners. All participants store unconfirmed transactions in their memory pools and confirmed transactions in their blockchains. Users create transactions by carefully choosing a fee and then diffuse them to be confirmed in a block in the blockchain. When miners receive a certain number of such transactions, they attempt to approve a selection of them as a block by executing a compute-intensive PoW algorithm (i.e., a hash-based proof of their work) of a predetermined difficulty. Notice that each miner tries to build a different block, using the transactions it received, placed into an order by itself. Upon finishing and executing the algorithm successfully, the corresponding miner transmits its block to the network to be appended to the blockchain. This block contains the transactions, the unique hash value (the solution of the puzzle), and a block reward, consisting of a static block reward plus the total fees of transactions. While the Bitcoin protocol is open source, the theoretical decoupling of the whole PoW algorithm is rather informal. Unfortunately, existing studies aimed at studying consensus focus only on the miner-related parts of the algorithm [3], [4] and omit its user-related parts. Consequently, it is crucial to precisely identify a formal model of the whole algorithm. Moreover, while some researchers classify PoW as a Nakamoto consensus algorithm [4]–[9] or as a proof-based (also called proof-of-X) consensus algorithm [10]–[15] or as a compute-intensive-based consensus algorithm [16], some other researchers do not even consider it as a consensus algorithm. In other words, there is no consensus about whether PoW is a consensus algorithm or not, and if it is, which type it is and what its general properties are. Besides, 22

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

JUNE 2022

none of these classifications enables us to identify how PoW can be improved and to what kind of environments it can be deployed. Contributions Based on these observations, we consider the consensus problem in blockchain systems from a coordination perspective, unlike the existing approaches. (We reported this idea initially in [17].) The contributions of this article are as follows: ●●  We provide a detailed formal model for the PoW algorithm-based blockchain systems, and based on it, we make a theoretical analysis from a coordination point of view. ●● We define PoW as a consensus algorithm and make a theoretical analysis of this definition from the point of views of the distributed systems (DSs) and multiagent systems (MASs). ●●  We show that PoW is a stigmergic consensus algorithm and identify its fundamental properties. ●● We propose a reengineered PoW to reveal its potential hidden strength thanks to stigmergy. To the best of our knowledge, there is no other study making similar contributions. PoW Blockchain System Model In this section, we delineate our notion of the basic PoW blockchain system model based on the Bitcoin protocol. This model is based on the Bitcoin source code (https://github.com/bitcoin/ bitcoin), bitcoin.org (https://bitcoin.org/), the Bitcoin Stack Exchange forum (https://bitcoin.stackexchange.com/), and the Bitcoin white paper [1]. Blockchain Network Model The blockchain network is modeled as a dynamic directed graph G = {N, E} where N denotes the dynamic node (vertex) set, and E denotes dynamic directed link (edge) set. A node n can join and quit G by using its join(G) and quit(G) actions, respectively. After joining G, n searches for neighboring nodes to connect to. Upon this process, directed links between n and other nodes are created. A directed link n " m is represented as a link {n, m} ! E where n, m ! N, n is the source of the link, and m is the neighbor of n. n can interact with a set of target nodes R n (where 6m ! R n ; {n, m} ! E) by sending/receiving messages of the form G n, msg, u H where n is the sender, msg is the type, and u is the data contained. Message exchanges are not instantaneous and with a delay of u e where e ! E is the link used for messaging. User Model A user node n has a list of its neighboring nodes N n where N n 3 N and 6m ! N n ; {n, m} ! E. Typically, a node n can manage up to eight outgoing neighbors and 117 incoming neighbors, even though in practice, those values can be modified. It uses addNeighbor(m) and removeNeighbor(m) actions to add and remove another node m as its neighbor, respectively. A three-tuple {m, # m, t m} is used to represent

each neighbor m ! N n where # m is the ban score and t m is the last communication time. t m is updated at each message receipt, and if m has not interacted for more than a predefined duration T timeout, n removes m from N n . The ban score # m is an integer value (0 # # m # 100), which is by default zero and is updated depending on the misbehaviors of m observed by n. (Misbehaviors of nodes are about transactions and blocks they diffuse to the network. A detailed list of rules for transactions and blocks can be found in https://en.bitcoin.it/wiki/Protocol_rules.) A misbehavior is detected using the actions of the form validateX ($) : # where X is the property to be validated and × is the ban value returned (0 # # # 100). An action validateX ($) returns zero if the validation succeeds and a value higher than zero otherwise (misbehavior detected). If # 2 0, then n updates # m using the updateBanscore (m, #) action where m is the misbehaving neighbor and × is the additional ban value. If # m crosses the ban score threshold x n of n, m is put in the Y n during a ban time d n defined by n banned neighbors list N (by default 24 h). Inside each node n, there is a memory pool H n, which is used to keep unconfirmed transactions that have input transr n, which is used to keep actions; there is an orphan pool H unconfirmed transactions that have one or more missing input transactions (orphan transactions); and there is a blockchain ledger B n, which is used to keep confirmed transacr n = 4, H n + B n = 4 and H r n + Bn = 4 tions where H n + H always hold. Miner Model We say that a node m is a miner node if it aims to create blocks using the unconfirmed transactions in its memory pool H m . It can begin and end this process using the startMining () and stopMining () actions, respectively. We denote the set of miner nodes by M where M 3 N. To mine, m ! M has to solve a computation-intensive puzzle (i.e., PoW [2]) using its computational hashing power q m where q m 2 0. Given a block header h i and a threshold n (i.e., the difficulty level), each miner repeatedly selects a nonce value h i until it gets a hash code lower than n with brute force. (The threshold limits the rate at which new blocks can be generated. In Bitcoin, the threshold is regularly adjusted to produce a block on average every 10 minutes. The advantage of this long period is that it dramatically reduces the number of simultaneous blocks mined by different miners.) The miner then creates an ith block b i and broadcasts it to its neighbors N m . If it succeeds appending its block to the blockchain of the other nodes, the miner is awarded by a fixed amount of reward plus the total fee of the transactions. Blockchain Data Structure Model The blockchain maintained by a node n is modeled as a r0 r1 dynamic append-only tree B n = {b 0 b1 rh - 1 f b h} where b 0 is the very first block (referred to as the genesis block), ; B n ; = h is the depth of B n, and b h (referred to as the blockchain head) is the furthest block from

b 0 and where each block b i (0 1 i # h) contains a cryptographic reference ri - 1 to its previous block b i - 1 . When a block b i - 1 has multiple children blocks, it causes a situation called a fork. In such a situation, the main branch is denoted as B *n and is identified as the longest path h from any block to b 0 where h = ; B *n ; and B *n 3 B such that ; B xn ; 1 ; B *n | for all branches B xn 1 B n where B xn ! B n* . All other branches are called side branches. If at any time, there are several longest paths with the same depth h (meaning that there are several head blocks), the blockchain B n is considered as inconsistent and thus B *n = 4. The blockchain becomes more consistent when a new block b h + 1 extends one of the side branches and creates B ∆n where ; B *n ; = h + 1. Block Model A block is denoted as b i = {h i, W i} where h i is the header of b i and W i is the data of b i . W i is a collection of all the transactions organized as a Merkle tree [18] where the copies of each transaction are hashed, and the hashes are then paired, hashed, paired again, and hashed again until a single hash remains, i.e., the root of the Merkle tree. A Merkle tree and its root are denoted as W i and } i, respectively. W i is created using the createMerkelTree (tx c, i m) action where m ! M is the miner node creating the Merkle tree, i m 3 H m is the set of selected transactions for the corresponding block, and tx c is the Coinbase transaction that rewards m with the block reward R = F + Rfi for its work (where F is the static block reward, and Rfi is the total fees of i m). A note to remember is that the coins in a Coinbase transaction cannot be spent until they have received 100 confirmations in the blockchain (which takes roughly 16 hours and 40 minutes). Note that the sizes of blocks are limited, and thus, the total size of i m cannot be more than this limit. The set of selected transactions i m is chosen using the selectTransactions (H m) : i m action where H m is the memory pool of m. Rf, the total fee, is then calculated by using calculateTotalFee (i m). The block header is denoted as h i = {H (h i - 1), H (h i), t bi, h i, } i} where H ( $ ) is the cryptographic hash function, h i - 1 is the header of the previous block b i - 1 (i 2 0), t bi is the time stamp for b i, and h i (called nonce) is a positive integer field whose value is to be found by m to generate the right H (h i), conforming to the difficulty level n. Transaction Model A transaction is denoted as tx = {I, O} where I is a list of inputs (I ! 4) in which each input i ! I references to a previous unspent output, called the unspent transaction output (UTXO), and O is a list of outputs (O ! 4). Upon being appended to the blockchain inside a block, each output o ! O waits as an UTXO until an input i spends it. If an output o has already been spent by i, it cannot be spent again by another input (no double spending [19], [20]). We co j, v pk} where Y co j $ 0) is the denote an output as o j = {Y co j (Y number of coins that this output will be worth when JUNE 2022

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

23

claimed, and v pk is a locking script that requires certain conditions to be met for a recipient m ! N to spend its coins. (Ideally, this script contains the public key of m so that m can unlock it using its private key.) We denote an input as i k = {H (tx k), p, v s} where H (tx k) is the hash of the previous transaction tx k, p is the index of the previous output, and v s is the unlocking script that satisfies the conditions placed on the output by v pk . As a rule, all inputs have to be spent within a transaction tx, and the total input coins Y cI have to be greater than or equal to the total output coins Y cO . The fee ftx of tx is f = c c . then denoted as tx YI YO In other words, any input coins not redeemed in an output are considered a transaction fee ftx; whoever generates the block can claim it by inserting it into the Coinbase transaction of that block. The fees to be spent for transactions are not predetermined and are estimated by using the estimateFee(I, O) action. Depending on the chosen fee, if there are still some unspent coins left, an output that pays this remainder to the sender itself can be added. tx c, the Coinbase transaction (see the “Block Model” section), is a unique transaction that spends all transaction fees, and exceptionally, it does not have any input set (I = 4). It is the very first transaction in a block and can be created only by the miner of that block. Transaction creation is modeled as createTransaction (m, Y c) where Y c is the amount of coins (Y c 2 0) paid to m (n, m ! N ). Selecting the right inputs to c) action spend Y c is modeled using the selectUTXOs (B n, Y B where n is the blockchain. PoW as a Consensus Algorithm There is a common, yet not grounded, belief in the blockchain community that PoW is a consensus algorithm [21]. Nearly every wiki, forum, blog, and article says, as if they are in agreement, that PoW is a consensus algorithm. Furthermore, some even group PoW and its similar counterparts, like the proof-of-stake (PoS) algorithm, the delegated PoS (DPoS) algorithm, and so on, under the umbrella of Nakamoto consensus algorithms (or proof-based or proof-of-X consensus protocols) [4]–[15]. However, these definition attempts provide in a very limited sense the general properties of systems using PoW. Literally, consensus relates to group decision-making processes where participants elaborate and agree on proposals with the objective, or the condition, of approval by all. In computer science, consensus algorithms are used to reach agreement on a common data value between distributed processes or systems. It is a fundamental problem in the DS and the MAS domains that aims to achieve overall system fault tolerance or resilience, respectively. Put into the blockchain context, consensus algorithms aim to agree on the new blocks (and thus the confirmed transactions and their orders) among the participants. In the following section, we make a theoretical analysis of this definition from the point of views of the aforementioned DS and MAS domains. 24

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

JUNE 2022

Consensus in DSs According to Tanenbaum and Steen [22], a distributed system is a set of independent computers (i.e., distributed processes) that behaves as a single coherent system for its users. In other words, as Leslie Lamport [56] says, “a distributed system is one in which the failure of a computer you did not even know existed can render your own computer unusable.” A fundamental problem in DSs is to guarantee the liveness and safety of the underlying system in the presence of a number of faulty system components with or without symptoms. Simply put, liveness guarantees that something good eventually happens, and safety should guarantee that something bad never happens. To this end, serious DSs support fault tolerance from the very basic level of abstraction against the following classes of failures (from basic to severe): crash, selfishness, attacking, and Byzantine. (A failure occurs whenever the process doesn’t behave as stated by the algorithm [23].) Byzantine failures are any failures presenting different signs to different processes [24] and are considered as the most difficult and most general category of the failures. In DSs, consensus algorithms aim to make all correct distributed processes decide exactly once (i.e., the finality is immediate; recall that, in the PoW algorithm, the finality is not immediate) on a common value that is chosen among the values that the processes initially propose despite any kind of faulty processes (derived from the definition of regular consensus given in [23]). In a nutshell, in these algorithms, regularly, a set of participants is selected as a committee. The whole process then evolves in rounds, and for each round, a committee member is selected as the proposer of a value to be agreed on (also called the leader). Note that the election of the leader is transparent, and the leader of the next round is known by all participants. Such a mechanism implies, in the case of blockchain systems (i.e., a system deployed to the Internet), having a closed system with communication with message passing to reach a consensus by tolerating faulty processes up to Byzantine failures. One can also think of considering the blockchain as a shared memory (or object) and propose using a model that is open (i.e., with an infinite number of participating processes). However, this requires one to assume that there is only one blockchain that is available directly to all the processes, and this is the case, for instance, for multithreaded machines where there is physical access to the shared memory. For blockchain systems, however, communication by messaging makes more sense. Consequently, in such consensus algorithms, a fixed-size committee of processes is selected (in which all processes know each other). One of them proposes a value to the committee members, the committee members exchange the proposed value with each other, they validate/invalidate it, and finally, if up to a certain majority of the committee members validate it, they decide it as a common value. This, obviously, requires direct interaction (i.e., communication) of the distributed processes with each other where interactions take

place via messages in which the identifier of the recipient is explicitly specified in messages. (Recall that, in PoW, new blocks are proposed to all processes/agents without specifying the identifier of the recipients.) Besides, given the difficulties of achieving consensus in dynamic and large-scale scenarios, DS proposes nondeterministic approaches (e.g., such as those based on gossiping [25] or probabilistic broadcast [26], [27]) or approaches based on eventual consistency [28]. Moreover, there are other proposals for distributed aggregation on sensor networks (e.g., the virtual macro sensor approach [29]) that ensure asymptotic convergence or reasonably approximate agreements on a value or that propose exploiting pheromone fields to reach agreement. However, as put forward by Gramoli [4], “it is not easy to relate the probabilistic guarantees offered by PoW consensus protocols to the consensus definition used in the DS literature.” Thus, the common approach of the DS community is either trying to define PoW as a randomized consensus algorithm with deterministic termination or not considering it as a consensus algorithm at all. Since they do not see PoW as a real consensus algorithm, they define possible attacks on PoW-based blockchains (e.g., Bitcoin) and discuss why PoW should be avoided [4]. However, all those possible attacks remain only possible, and Bitcoin, for example, has operated without any major problems since 2008. Consensus in MASs An MAS is a loosely coupled structure composed of multiple agents where they try to solve problems that cannot be tackled individually (due to lack of ability, knowledge, or resources or even low-efficiency problems) by interacting with each other [30]. An essential problem in MAS is to guarantee resilience in 1) time-varying connectivity, 2) lack of centralized control and access to information, and 3) indirect interaction between agents. Simply put, resilience should guarantee robustness against changes in the system. In MASs, consensus algorithms aim to make agents decide asymptotically (i.e., the finality is not necessarily immediate) on a common value that is based on the values that the agents propose through local interactions and computations. (This definition is based on the asynchronous asymptotic agreement problem for parallel computing and distributed decision-making systems [31].) There are basically two types of agreement in MAS consensus, either by agreeing on one of the proposed values (e.g., bees deciding on a food source [32] or ants deciding on a trail for a food source [33]) or by agreeing on a fusion of all proposed values (e.g., time synchronization in a wireless sensor network [34] or collective behavior of flocks and swarms [35]). Here, it should be noted that, in the former case, since the number of possible alternative values may increase dramatically, the agents may include mechanisms to reduce this number. (Recall that, in PoW, to reduce the number of alternative blocks significantly, the agents are forced to solve a very hard computational puzzle.)

Besides, consequently and not surprisingly, MAS consensus algorithms use indirect coordination where agents can coordinate by indirectly interacting with each other through persistent and perceivable changes to a common environment where recipients are all agents who will perceive these changes [36]. Environment is a first-class abstraction in the MAS paradigm [37]; i.e., it is a participant and a shared memory, not just a medium for interaction. Examples of indirect interaction in MAS include the interaction of processes using semaphores, interaction of social insects using pheromones, interaction of dining philosophers via chopsticks, interaction of buyers and sellers through a market, information exchange of cells via hormones in the bloodstream, and so on. Considering a blockchain system as an MAS, the environment is the shared data structure blockchain. The blockchain is a persistent, dynamic, and virtual environment for the agents. It allows agents to interact indirectly with each other and aggregates the information. PoW and Stigmergic Consensus In this section, we further elaborate our theoretical analysis of the dynamics of PoW in more detail and show that PoW is a stigmergic consensus algorithm. PoW as a Stigmergic Algorithm The concept of stigmergy was first introduced to shed light on the construction of termite colonies [38]. It explained, for the first time, how apparently independent and random movements of an individual could result in the generation of coordinated behavior at a global level by persistently transferring information locally [33]. Up until now, stigmergy has been used to describe a wide range of collective behaviors, such as ants deciding on a trail for a food source, herd migration in animals, humans following hiking trails and pedestrian footpaths, and swarm intelligence in computing. Next, we analyze the dynamics of PoW on a case inspired by a well-known stigmergic MAS consensus mechanism to be able to define the general properties of systems using PoW. In this sense, we have chosen ants deciding on a trail for a food source. The objective of an ant society is to bring back as much food as possible from a food source that will satisfy its members [Figure 1(a)]. When worker ants leave their nest to find food (i.e., foraging), they leave behind a trail of pheromones (resembling a dotted line of chemicals) along the way. When they find food, these ants fill up their social stomachs with as much food as they can. Then, the ants scurry back to their nests, following the trail, again leaving pheromones along the way. The ants will begin sharing food with other members of their colony through their social stomach, and more ants will begin to follow the traced scent back to the food source. Every time an ant visits the food source, it adds to the effectiveness of the scent trail. However, their orchestration is still poor because very little pheromone is on the trail. In the meantime, certain other ants may take various ways to the food source and back to the nest, again leaving traces of scent. This JUNE 2022

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

25

Nest

Food Source

(Eventually Abandoned)

Nest

Food Source

(Eventually Abandoned) Distance (a)

Genesis Block

All Transactions Confirmed

(Eventually Abandoned)

Genesis Block

All Transactions Confirmed

(Eventually Abandoned) Height (b)

Figure 1. (a) An ant society versus (b) a PoW-based blockchain system. In (a), some pathways are eventually abandoned with the attenuation of pheromones with time. In (b), the chains that do not reach a certain depth are eventually abandoned with the attenuation of blocks with time (i.e., these blocks become less attractive, and thus, their probability to get deeper decreases exponentially [3]). Note that while in (a), the pheromones completely disappear after some time, in (b), the blocks stay.

26

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

JUNE 2022

eventually leads to an optimization of the path; since pheromones are evaporative, the shorter the trail is, the stronger the scent is—so more ants take the shortest trail, again leaving scent traces. Now, consider a PoW-based blockchain system (e.g., Bitcoin) like an ant society [Figure 1(b)]. The objective of such a society is to confirm as many transactions as possible to satisfy its members. The food source resembles the state in which all (including future) transactions are confirmed, and bringing a part of the food to the nest resembles confirming some unconfirmed transactions. When a block creator agent (i.e., miner) finds unconfirmed transactions, it fills up its new block with as many transactions as it can. Then, it appends its block to the blockchain and creates a trail of blocks linked by cryptographic links. This way, it shares confirmed transactions with other members of its society, and more miners begin to follow the traced scent (i.e., the longest branch) to reach the state where all transactions are confirmed. Every time a miner confirms transactions as a block, it adds to the effectiveness of the branch. Meanwhile, some other miners may take different ways (i.e., branches), again leaving trails of blocks. This eventually leads to an optimization of the path; since cryptographic links are very strong, the main branch becomes the stronger as it becomes longer—so more miners follow the longest branch, again leaving blocks. Just like in an ant society, where individuals work as if they were alone while their collective activities appear to be coordinated [33], in a PoW-based blockchain system, agents work as if they were alone while their collective activities appear to be coordinated. This is called stigmergy: an indirect, mediated mechanism of coordination between actions, in which the trace of an action left on a medium stimulates the performance of a subsequent action [39]. Stigmergy fits well to PoW since the blocks adopted by agents in the system guide which next blocks will be created and which next transaction will be issued. Consequently, we can conclude that PoW is a stigmergic consensus algorithm. The Fundamental Properties of PoW as a Stigmergic Consensus Algorithm Stigmergy enables complex, coordinated activity without any need for planning, memory, communication, mutual awareness, simultaneous presence, imposed sequence, commitment, or supervision [39]. This fits well to PoW-based blockchain systems. ●● Planning: Agents should be conscious only of the current state of the operation. There is no strategy or plan defining which block (or transaction) needs to be appended to the blockchain at what time. ●● Memory: Agents do not need to recall their previous activities; there is no need to store information about the state of the work anywhere but in the blockchain. Thanks to this property, agents can easily exit a blockchain system and then come back again.

Communication: No information for negotiating over the actions to be taken needs to be exchanged between the agents (see the “Blockchain Network Model” section), except through the work performed in the blockchain. ●● Mutual awareness: Every agent operates independently; they do not even need to know that others are present in the system to decide which actions to be taken. Of course, as stated in the “User Model” section), agents keep track of their neighbors and may decide to ban them if necessary. However, at this level, agents do not know the identities of their neighbors, and their ban action is simply about the relay of the messages. ●● Simultaneous presence: The agents need not be present simultaneously; agents can work whenever and wherever they are available, thanks to the fact that the traces (i.e., blocks and transactions in them) are persistent and can guide agents at any later time. ●● Imposed sequence: Actions are carried out inherently in the right order since an action would not be commenced until the right condition is in place; the workflow emerges as the finishing of one block creation or the creation of the next or as the confirmation of a transaction triggers the creation of the successive transactions. ●● Commitment: Agents do not need to commit to the creation of a specific block. Agents choose instantly what actions they should perform, depending on incentive and other contingent conditions; agents that quit or otherwise become unavailable are spontaneously replaced by other agents. ●● Supervision: Disruptions (e.g., branches) are automatically fixed, without any centralized control directing the activity. In the stigmergic model, by the coordination indicated in stigmergically orchestrated acts, the social good (e.g., a consistent blockchain) is increasingly built up. Any agent may then profit from this common good without putting in any effort in return without reducing its value. An agent without leaving a stigmergic trace (i.e., blocks or transactions) does not by that action make the blockchain less useful to the other agents. In fact, by validating all the data it receives and diffusing only the ones that are valid, it increases the quality of the stigmergic trace and thus the security of the whole system. The trace in the blockchain is both an individual and a collective mental map that indicates the effective actions of the agents, and not leaving a trace may make their own future work harder than it needs to be. For instance, having a transaction included in the blockchain implicitly tells the creator of that transaction the information, like approximately how long it took for the transaction to be included in the blockchain and whether or not the fee put by the creator was good enough. ●●

Reengineering PoW PoW was not initially engineered as a stigmergic algorithm. Consequently, its entire potential strength remains unused. What is missing is that in the original PoW, the blockchain JUNE 2022

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

27

and the memory are mere containers of transactions and blocks, and thus, they lack environmental mechanisms for a better coordination. Engineering Principles Before starting to reengineer PoW as a stigmergic algorithm, we need to identify the engineering principles specific to this kind of system. Unlike other stigmergic systems, PoW blockchains are deployed to the Internet, and they should be tolerant to any kind of fault. To this end, we identified the following key principles: ●● The agents cannot inherently rely on (i.e., take into account without questioning) the information coming from the others. ●● The agents can inherently rely on their own local information (e.g., the ban score and their own local clocks). ●● The agents can take into account the information found in their local blockchains directly (e.g., confirmed transactions and blocks). ●●  The agents can take into account the information validatable on their local blockchains (e.g., an unconfirmed transaction that is using the UTXOs on the blockchain and an unconfirmed block whose hash code is successively linked to other blocks). In the following, based on these principles, we reengineer the PoW algorithm.

Amplitude of Response

Digital Pheromones To add stigmergic environmental mechanisms to the blockchain and the memory pool, we propose the usage of digital pheromones. Basically, digital pheromones generated by intelligent agents are virtual and can represent more information than chemical pheromones left by different ants. Utilizing transactions and blocks as digital pheromones can improve the effectiveness of stigmergic PoW [40]. This way, the blockchain can distribute digital pheromones for providing relevant information of the state space. Similar to the natural phenomena, here the objective is to create the most attractive branch. To this end, we model the accumulation of digital pheromones with different sources as a linear superposition and the decay of the fixed amount of digital pheromone over time. Note that since

•

Attractors When a digital pheromone r is appended to the local blockchain of an agent, it becomes an attractor. Attractors can coexist in parallel on the different branches of a local blockchain B n and can have different values for each branch B kn . When a digital pheromone r is appended to B n by linking to an attractor, a maximum pheromone value t max is assigned to the corresponding pheromone level parameter t r . Afterwards, the pheromone level t r of attractors is gradually decreased by a discount constant d where 0 1 d 1 1. If at any time the pheromone value t kr decreases below zero and if the attractor r does not have any linked attractors, it is erased from B kn using the action of the form evaporate (r, B kn). Attractors have an attractive effect on agents for efficiently creating their new digital pheromones. Using their local blockchains, each agent can independently create their digital pheromones by selecting suitable attractors from several potential attractor candidates. That is, while a user agent can issue a new transaction by linking its inputs to the highest UTXOs of the most attractive blockchain branch, a miner agent can create a new block by linking it to the head block of the most attractive blockchain branch. Recall that the overall objective is to reach the state where all transactions are confirmed. This mechanism can be expressed by

1 0.8 0.6 0.4 0.2

U n, b, a =

A (d b, a (t)) $ { ba (t) / A (d b,a (t)) $ { ba (t) , (1)

a ! T bn (t)

0

10

20 30 Height of Attractor

40

50

Figure 2. The amplitude of response of an agent n to the attractors in its local blockchain B in with respect to their depth (i.e., A ( $ )).

28

the objective is to have a single main branch in the blockchain, the digital pheromones should be released only into a single branch. Based on the aforementioned engineering principles, the initial pheromone level of each digital pheromone is assigned by the time of their receipt rather than diffusion. Concretely, when an agent n receives a digital pheromone r, it first stores the acceptance time t r of pheromone r by retrieving the local time from its own local clock using the action of the form getLocalTime () and also assigns a maximum pheromone value t max to its corresponding pheromone level parameter t r . Then, agent n stores r in its memory pool H n, as described in the “User Model” section. Afterwards, the pheromone level t r is gradually decreased by a decay rate d where 0 1 d 1 1. If at any time the pheromone value t r decreases below zero, the pheromone r is erased from H n using the action of the form evaporate (r, H n).

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

JUNE 2022

where U n, b, a is the probability of agent n selecting attractor a in blockchain branch B bn; { ba (t) is the total amount of digital pheromone in attractor a at time t; T bn (t) is the set of attractors in B bn; d b, a (t) is the depth of an attractor a in branch B bn; and A (·) is a monotone function used to reduce the effect of the digital pheromone as the interdistance d b, a (t) increases (Figure 2).

This attractor selection method is inspired by the classical Ant Colony Optimization algorithm [41]. Function A (·) can cause an agent to pay more attention to shallower influential attractors in a chosen branch B in . However, A (·) cannot, for instance, avoid the confirmation of a transaction to and from between different branches, which occurs when there are several blocks confirming the same transaction due to the race between branches. To avoid such situations, we modify the input of the spending transaction i k as in (2). i k = {H (h i), H (tx k), p, v s}, (2)



where H (h i) is the header of the block b i of the corresponding UTXOs to the input of the spending transaction. Block Finalization In basic PoW (described in the “PoW Blockchain System Model” section), the process of finalizing a block b i involves validating the block header h i . In the reengineered PoW, the validation of the header h i is very similar to PoW and means verifying that the block nonce h i is correct and that each transaction in the Merkle tree W i is valid, satisfying that the depth of the block b i to the present head block b h is h i $ 1. Formally: isValid (W i) / isKin (b i, b h, 1), (3)

where

isValid (W i) = *

true, if

/

tx ! W i

validateTX (tx) = 0

false, otherwise

, (4)

isKin (b i, b h, n) / h - i $ n, (5) where validateTX(tx) denotes the action for validating transaction tx (see the “Blockchain Network Model” section) and isKin denotes is-kin property. Analyses In this section, we make an analysis of PoW and reengineered PoW (R-PoW). In PoW, only the miners can choose the chain to extend, and they do so with respect to the length of the chains (i.e., the longest chain rule). In a case where there is more than one longest chain with the same length, a miner first tries to choose the chain that has more blocks; otherwise, it chooses one of the longest chains randomly. However, in R-PoW, this decision is not only up to the miners. The miners also have to choose a chain to extend with respect to the preferences of the users (i.e., depending on the chain where they chose their input from). Moreover, in a realistic setting where the system deployed to the Internet, even if the lengths of the chains are the same, the probability of having the same attractivity in these chains is low (i.e., the probability of having the same number of transactions is low) due to the package losses and message delays in the network. Consequently,

in R-PoW, the miners have a much narrow chain selection option (usually only one); in other words, the number of possible forks is lower. In PoW, any chain can be forked from anywhere at any time. This makes both selfish-mining attacks [2] (also called block withholding attacks) and rewriting of the whole blockchain (which is technically not impossible) possible. However, in R-PoW, the deeper a block is, the harder it is to fork it (due to the decay of pheromone levels). Consequently, in R-PoW, it is not convenient to try to make selfish-mining attacks because, as the attacker is silently mining blocks by itself, the users are issuing their transactions by binding them to the released blocks. And it is not possible to rewrite the blockchain from scratch by a miner since the released transactions already define which chains they are bound to. Discussion To the best of our knowledge, there is no other study categorizing PoW as a stigmergic consensus algorithm. Considering that the blockchain is a shared memory structure, albeit with interesting safety guarantees, the only studies we are aware of are [42], which proposes the idea that shared memory structures can be used as virtual stigmergy for swarms of robots, and [43], which proposes a language for describing virtual stigmergy. However, both studies do not take into account a system like blockchain that can be deployed to the Internet and that can be subject to severe failures like selfishness [44], [45] and Byzantine failures [9]. Besides, Capaccioli [46] tries to create a model for Bitcoin based on swarm intelligence and stigmergy to help understand the dynamics of blockchains and how regulation could affect different blockchain actors (i.e., the effectiveness of possible rules). Likewise, it also makes an analogy to an ant society following a path and says that the actors are following virtual pheromones. However, the differences of [46] from our study are manyfold: 1) It does not show how the actors are following virtual pheromones; however, we show it by providing a detailed formal model of the underlying protocol. 2) It does not identify the fundamental stigmergic properties and how well they fit to PoW. 3) It does not show how to reengineer such a system to make a better stigmergic system. 4) It focuses on the higher-level actions about the governance of the blockchain system; however, we focus on the lowerlevel actions about the construction of the blockchain data structure. 5) It identifies that the pheromone is contained in the Bitcoin core source code; however, we explicitly define that the pheromones are the transactions and blocks and formally define their attractor functions. 6) It claims that the essential actors (i.e., users, miners, nodes, and developers) cannot deviate from the code; however, as we have shown from the literature, there can be malicious actors that are deviating from the default protocol. JUNE 2022

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

29

Importance of User Agents Although it is not a disruptive innovation, a full modeling of the PoW protocol with both its users and miners was actually missing, to the best of our knowledge. The PoW system model given in the “PoW Blockchain System Model” section allowed us to clarify the importance of the role of user agents in the PoW algorithm and how their active participation can improve it. Intuitively, Hou and Chen [47] came to the same conclusion that we need to change the role of users in the system. The users should not “use” only the Bitcoin system; they should have certain capabilities to determine which miners can process transactions and gain the incentives. This way, we may give more rights to users and constrain the freedom of miners. To this end, they propose a user-determined rewarding mechanism where, for example, the users can rank (by voting) a miner based on its processing history. Another solution with the same idea is the DPoS protocol (https:// en.bitcoinwiki.org/wiki/DPoS), where users are allowed to evaluate and select miners. This way, the miners are incentivized to make more user-centered decisions. However, such solutions are problematic in two senses: 1) As they also pointed out, we may need additional mechanisms to prevent users from abusing their rights and to differentiate the weights of different users’ votes. 2) The identities of the miners should be known, and consequently, miners can continuously change their accounts to avoid being low ranked. The reengineered PoW (see the “Reengineering PoW” section) does indirectly constrain the freedom of miners (without needing to know their identities). In the PoW-based blockchains, as well as in the aforementioned solutions, the miners are free to choose to which block to append their newly created blocks, and consequently, the miners are free to fork a new branch from any point of the blockchain. In our solution, however, the users decide which UTXOs to spend from which specific blocks, i.e., transaction instances are directly linked to each other. The direct linkage of transactions is expressed in (2). The effect of this improvement is that the user agents have to choose a unique transaction instance located inside a single block in case there are several blocks containing the different instances of the same transaction. Since according to (1), user agents choose the block, the miner agents should then also take into account this information (in addition to the fees) to be able to decide which chain to extend in case of forks. This would eventually reduce (if not remove) the forks, and the good thing is that choosing the chain to extend will not be merely decided by the miners but collectively by all agents. In fact, the agents can even accomplish a hard fork (in the case of an attack to the blockchain) in a collective manner (which normally can be decided and accomplished only by the developers). The secondary effect of this improvement is that knowing the exact block b i may reduce this excessive processing cost for 30

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

JUNE 2022

validation since UTXOs can be found much more quickly. (The expensive element of transaction validation is the verification of the script [48].) Currently, there are around 100 million of UTXOs in Bitcoin (https://bitcoin.stackexchange. com/questions/91271/how-long-does-it-take-to-find-a -utxo-in-utxos-set). In general, the set of UTXOs can be stored in memory as a list keyed by the UTXO identifier, and this provides O (1), or constant, lookup complexity for a single UTXO. However, the UTXO set may not fit entirely in memory on many systems (taking into account also the forked branches), and the actual retrieval time may be longer than expected. Decay of Pheromone Levels Stigmergic systems commonly maintain information by continuously refreshing current information and letting obsolete information evaporate [49]. This process guarantees that inconsistencies remove themselves within a specified time horizon, without the need for complex truth-maintenance procedures. Note that this is different from pruning [50], where the objective is to reduce the disk space typically required to host a copy of the blockchain. Evaporation is a stigmergic action that stimulates and/or affects subsequent actions. However, while in an ant society, evaporation is accomplished by the environment itself, in a PoW blockchain system, it should be managed by the agents since they manage the local replications of their blockchains. The implications of the decay of the digital pheromone level are manyfold. It will dramatically reduce the probability of miner block withholding attacks [44] where a miner suddenly releases several blocks to change the main branch. It will gradually reduce the storage needed to persistently maintain a full blockchain copy, which is also necessary to help bootstrap joining nodes [51]. It will require lower bandwidth (which is especially good for joining nodes to obtain a full blockchain copy [51]), and it will have a lower processing cost thanks to the absence of large numbers of transactions remaining on the forked branches. Self-Organizing Properties of PoW In the “The Fundamental Properties of PoW as a Stigmergic Consensus Algorithm” section, we have shown that the basic PoW algorithm holds the fundamental stigmergic properties. Moreover, it also holds some self-* properties [52], such as self-healing and self-resilience. Self-healing property represents the ability to recover under failures. Recently, it was reported that PoW is self-healing, while the Byzantine fault tolerance algorithm is not [53]. Self-resilience property represents the ability to reliably provide a service while there are failures. In DS consensus algorithms (see the “Consensus in DSs” section), the fact that the creators of the future blocks are known makes the blockchain open to attacks. To overcome such situations, the DS community proposes randomized solutions [54]. In PoW, however, it is already not possible to know which miners will be the leaders of future blocks.

Conclusions and Prospects In this article, we have shown that PoW is a stigmergic consensus algorithm. Agents participating in a PoW blockchain coordinate their actions by relying at some level on stigmergy, in the sense that subsequent actions are stimulated by the trace left by previous actions in an observable and manipulable (append-only) medium, i.e., the blockchain. The traces in the blockchain indicate which actions have been performed and which can still be performed. This information is shared by all agents that have access to the blockchain (i.e., successfully obtained or maintained a full blockchain copy), thus allowing them to coordinate their actions. Consequently, a PoW blockchain is a sort of collective intelligent system [52] mechanism in the sense that it relies on feedback; action elicits action, through the intermediary of the blockchain (i.e., the trace). Typically, this feedback is positive, with actions intensifying and elaborating the blockchain, thus eliciting more intense and diverse further actions. The resulting cycle enables the construction of blockchains. This conclusion enables us to identify what type of system PoW is creating, how it can be improved, and where it can be deployed (e.g., to the Internet, to robot swarms. or to wireless sensor networks, where, for instance, mutual awareness and simultaneous presence are not mandatory). To unlock the full potential of PoW as a stigmergic algorithm, as a future work, we plan to model agents as cognitive entities that are goal/task driven, and thus, they can typically be rational, heterogeneous, adaptive, and capable of learning. We will also identify artifacts to fully model this cognitive stigmergy. Such improvements can make, for example, the most well-known PoW blockchain Bitcoin a better collective intelligent system operating on the Internet. This is an important prospect since it was recently envisioned that the conditions for collective intelligence (CI) will be mature in the 6G era, and CI will emerge among the widely connected beings and things [55]. Considering that Bitcoin is a living successful example, it is natural to conclude that there will be a potential huge impact of PoW blockchains on boosting both communications and intelligence during the near future. References [1] S. Nakamoto. “Bitcoin: A peer-to-peer electronic cash system.” Bitcoin. https://bitcoin.org/bitcoin.pdf (Accessed: Apr. 15, 2022). [2] A. Back, “Hashcash–A denial of ser vice counter-measure,” Tech. Rep., 20 02 . [Online]. Available: http://www.hashcash.org/ papers/hashcash.pdf [3] J. Garay, A. Kiayias, and N. Leonardos, The Bitcoin Backbone Protocol: Analysis and Applications. Berlin, Germany: Springer-Verlag, 2015, pp. 281–310. [4] V. Gramoli, “From blockchain consensus back to byzantine consensus,” Future Gener. Comput. Syst., vol. 107, pp. 760–769, Jun. 2020, doi: 10.1016/j.future.2017.09.023. [5] J. Bonneau, A. Miller, J. Clark, A. Narayanan, J. A. Kroll, and E. W. Felten, “SoK: Research perspectives and challenges for bitcoin and cryptocurrencies,” in Proc. IEEE Symp. Security Privacy, May 2015, pp. 104–121, doi: 10.1109/SP.2015.14.

[6] L. Luu, J. Teutsch, R. Kulkarni, and P. Saxena, “Demystifying incentives in the consensus computer,” in Proc. 22nd ACM SIGSAC Conf. Comput. Commun. Security, New York, NY, USA, 2015, pp. 706–719, doi: 10.1145/2810103.2813659. [7] N. Stifter, A. Judmayer, P. Schindler, A. Zamyatin, and E. R. Weippl, “Agreement with Satoshi - on the formalization of Nakamoto consensus,” IACR Cryptology ePrint Arch., vol. 2018, p. 400, May 2018. [8] R. Zhang and B. Preneel, “Lay down the common metrics: Evaluating proof-of-work consensus protocols’ security,” in Proc. 2019 IEEE Symp. Security Privacy, pp. 175–192, doi: 10.1109/SP.2019.00086. [9] C. Cachin and M. Vukolic, “Blockchain consensus protocols in the wild,” in Proc. 31 Int. Symp. Distrib. Comput. (DISC), 2017, pp. 1:1–1:7. [Online]. Available: https://drops.dagstuhl.de/opus/volltexte/2017/8016/pdf/ LIPIcs-DISC-2017-1.pdf [10] U. Bodkhe, D. Mehta, S. Tanwar, P. Bhattacharya, P. K. Singh, and W. Hong, “A survey on decentralized consensus mechanisms for cyber physical systems,” IEEE Access, vol. 8, pp. 54,371–54,401, Mar. 2020, doi: 10.1109/ACCESS.2020.2981415. [11] E. K. Wang, R. Sun, C.-M. Chen, Z. Liang, S. Kumari, and M. Khurram Khan, “Proof of x-repute blockchain consensus protocol for IoT systems,” Comput. Security, vol. 95, p. 101,871, May 2020, doi: 10.1016/j.cose.2020.101871. [12] S. Bano et al., “SoK: Consensus in the age of blockchains,” in Proc. 1st ACM Conf. Adv. Financial Technol., New York, NY, USA, 2019, pp. 183– 198, doi: 10.1145/3318041.3355458. [13] W. Wang et al., “A survey on consensus mechanisms and mining strategy management in blockchain networks,” IEEE Access, vol. 7, pp. 22,328–22,370, Jan. 2019, doi: 10.1109/ACCESS.2019.2896108. [14] N. Chaudhry and M. M. Yousaf, “Consensus algorithms in blockchain: Comparative analysis, challenges and opportunities,” in Proc. 2018 12th Int. Conf. Open Source Syst. Technol. (ICOSST), pp. 54–63, doi: 10.1109/ICOSST.2018.8632190. [15] G.-T. Nguyen and K. Kim, “A survey about consensus algorithms used in blockchain,” J. Inf. Process. Syst., vol. 14, no. 1, pp. 101–128, 2018. [16] L. Ismail and H. Materwala, “A review of blockchain architecture and consensus protocols: Use cases, challenges, and solutions,” Symmetry, vol. 11, no. 10, p. 1198, 2019, doi: 10.3390/sym11101198. [17] Ö Gürcan, “Proof-of-work as a stigmergic consensus algorithm: Extended abstract,” in Proc. 21st Int. Conf. Auton. Agents Multiagent Syst., 2022, to be published. [18] R. C. Merkle, A Digital Signature Based on a Conventional Encryption Function. Berlin, Germany: Springer-Verlag, 1988, pp. 369–378. [19] C. Pinzón and C. Rocha, “Double-spend attack models with time advantange for bitcoin,” Electron. Notes Theor. Comput. Sci., vol. 329, pp. 79–103, Dec. 2016, doi: 10.1016/j.entcs.2016.12.006. [20] G. Karame, E. Androulaki, M. Roeschlin, A. Gervais, and S. Capkun, “Misbehavior in Bitcoin: A study of double-spending and accountability,” ACM Tran. Inf. Syst. Security, vol. 18, no. 1, pp. 1–32, Jun. 2015, doi: 10.1145/2732196. [21] F. Tschorsch and B. Scheuermann, “Bitcoin and beyond: A technical survey on decentralized digital currencies,” IEEE Commun. Surveys Tuts., vol. 18, no. 3, pp. 2084–2123, 2016, doi: 10.1109/COMST.2016.2535718. [22] A. S. Tanenbaum and M. V. Steen, Distributed Systems: Principles and Paradigms, 2nd ed. Englewood Cliffs, NJ, USA: Prentice-Hall, 2007. [23] C. Cachin, R. Guerraoui, and L. Rodrigues, Introduction to Reliable and Secure Distributed Programming, 2nd ed. Cham: Springer International Publishing, 2011.

JUNE 2022

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

31

[24] K. Driscoll, B. Hall, M. Paulitsch, P. Zumsteg, and H. Sivencrona, “The real byzantine generals,” in Proc. 23rd Digit. Avionics Syst. Conf., Oct. 2004, vol. 2, pp. 6.D.4–61, doi: 10.1109/DASC.2004.1390734. [25] M. Jelasity, A. Montresor, and O. Babaoglu, “Gossip-based aggregation in large dynamic networks,” ACM Trans. Comput. Syst., vol. 23, no. 3, pp. 219–252, Aug. 2005, doi: 10.1145/1082469.1082470. [26] R. Guerraoui, P. Kuznetsov, M. Monti, M. Pavlovic, and D. Seredinschi, “Scalable byzantine reliable broadcast,” in Proc. 33rd Int. Symp. Distr. Comp., J. Suomela, Ed. Budapest, Hungary: Schloss Dagstuhl, 2019, vol. 146, pp. 22:1–22:16. [27] P. T. Eugster, R. Guerraoui, S. B. Handurukande, P. Kouznetsov, and A.-M. Kermarrec, “Lightweight probabilistic broadcast,” ACM Trans. Comput. Syst., vol. 21, no. 4, pp. 341–374, Nov. 2003, doi: 10.1145/945506.945507. [28] S. Dubois, R. Guerraoui, P. Kuznetsov, F. Petit, and P. Sens, “The weakest failure detector for eventual consistency,” in Proc. 2015 ACM Symp. Princ. Distrib. Comput., Association for Computing Machinery, New York, NY, USA, pp. 375–384, doi: 10.1145/2767386.2767404. [29] N. Bicocchi, M. Mamei, and F. Zambonelli, “Self-organizing virtual macro sensors,” ACM Trans. Auton. Adaptive Syst., vol. 7, no. 1, pp. 1–28, 2012, doi: 10.1145/2168260.2168262. [30] E. H. Durfee, V. R. Lesser, and D. D. Corkill, “Trends in cooperative distributed problem solving,” IEEE Trans. Knowl. Data Eng., vol. 1, no. 1, pp. 63–83, 1989, doi: 10.1109/69.43404. [31] D. P. Bertsekas and J. N. Tsitsiklis, Parallel and Distributed Computation: Numerical Methods. Englewood Cliffs, NJ, USA: PrenticeHall, 1989. [32] T. D. Seeley, The Wisdom of the Hive. Cambridge, MA, USA: Harvard Univ. Press, 1995. [33] S. Camazine, N. R. Franks, J. Sneyd, E. Bonabeau, J.-L. Deneubourg, and G. Theraula, Self-Organization in Biological Systems. Princeton, NJ, USA: Princeton Univ. Press, 2001. [34] K. S. Yildirim and Ö. Gürcan, “Efficient time synchronization in a wireless sensor network by adaptive value tracking,” IEEE Trans. Wireless Commun., vol. 13, no. 7, pp. 3650–3664, Jul. 2014, doi: 10.1109/ TWC.2014.2316168. [35] R. Olfati-Saber, “Flocking for multi-agent dynamic systems: Algorithms and theory,” IEEE Trans. Autom. Control, vol. 51, no. 3, pp. 401– 420, 2006, doi: 10.1109/TAC.2005.864190. [36] D. Keil and D. Goldin, “Indirect interaction in environments for multi-agent systems,” in Environments for Multi-Agent Systems II, D. Weyns, H. Van Dyke Parunak, and F. Michel, Eds. Berlin, Germany: Springer-Verlag, 2006, pp. 68–87. [37] D. Weyns, A. Omicini, and J. Odell, “Environment as a first class abstraction in multiagent systems,” Auton. Agents Multi-Agent Syst., vol. 14, no. 1, pp. 5–30, Feb. 2007, doi: 10.1007/s10458-006-0012-0. [38] P.-P. Grassé, “La reconstruction du nid et les coordinations interindividuelles chezbellicositermes natalensis et cubitermes sp. la théorie de la stigmergie: Essai d’interprétation du comportement des termites constructeurs,” Insectes Sociaux, vol. 6, no. 1, pp. 41–80, 1959, doi: 10.1007/BF02223791. [39] F. Heylighen, “Stigmergy as a universal coordination mechanism I: Definition and components,” Cogn. Syst. Res., vol. 38, pp. 4–13, Jun. 2016, doi: 10.1016/j.cogsys.2015.12.002. [40] H. Van Dyke Parunak, S. A. Brueckner, and J. Sauter, “Digital pheromones for coordination of unmanned vehicles,” in Environments for

32

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

JUNE 2022

Multi-Agent Systems, D. Weyns, H. Van Dyke Parunak, and F. Michel, Eds. Berlin, Germany: Springer-Verlag, 2005, pp. 246–263. [41] M. Dorigo, M. Birattari, and T. Stutzle, “Ant colony optimization,” IEEE Comput. Intell. Mag., vol. 1, no. 4, pp. 28–39, 2006, doi: 10.1109/ MCI.2006.329691. [42] C. Pinciroli, A. Lee-Brown, and G. Beltrame, “A tuple space for data sharing in robot swarms,” EAI Endorsed Trans. Collaborative Comput., vol. 2, no. 9, pp. 1–8, May 2016, doi: 10.4108/eai.3-12-2015.2262503. [43] R. De Nicola, L. D. Stefano, and O. Inverso, “Multi-agent systems with virtual stigmergy,” Sci. Comput. Program., vol. 187, p. 102,345, Feb. 2020, doi: 10.1016/j.scico.2019.102345. [44] I. Eyal and E. G. Sirer, “Majority is not enough: Bitcoin mining is vulnerable,” in Proc. Int. Conf. Financial Cryptography Data Security, Springer-Verlag, 2014, pp. 436–454, doi: 10.1007/978-3-662-45472-5_28. [45] A. Sapirshtein, Y. Sompolinsky, and A. Zohar, “Optimal selfish mining strategies in bitcoin,” in Proc. Int. Conf. Financial Cryptography Data Security. Springer-Verlag, 2016, pp. 515–532, doi: 10.1007/978-3-66254970-4_30. [46] S. Capaccioli, “Blockchain, Bitcoin and stigmergy: An explanation and a new perspective for regulation,” BioLaw J. – Rivista di BioDiritto, vol. 2, no. 2020, pp. 467–476, 2020. [47] B. Hou and F. Chen, “A study on nine years of bitcoin transactions: Understanding real-world behaviors of bitcoin miners and users,” in Proc. 40th IEEE Int. Conf. Distrib. Comput. Syst. (ICDCS’20), Singapore, 2020, pp. 1031–1043, doi: 10.1109/ICDCS47774.2020.00091. [48] V. Vallois and F. A. Guenane, “Bitcoin transaction: From the creation to validation, a protocol overview,” in Proc. 1st Cyber Securit y Netw. Conf. (CSNet), 2017, pp. 1–7, doi: 10.1109/CSNET.2017. 8241988. [49] H. Van Dyke Parunak and S. A. Brueckner, Engineering Swarming Systems. Boston, MA, USA: Springer Science & Business Media, 2004, pp. 341–376. [50] E. Palm, O. Schelén, and U. Bodin, “Selective blockchain transaction pruning and state derivability,” in Proc. 2018 Crypto Valley Conf. Block. Tech. (CVCBT), pp. 31–40, doi: 10.1109/CVCBT.2018.00009. [51] R. Matzutt, B. Kalde, J. Pennekamp, A. Drichel, M. Henze, and K. Wehrle, “How to securely prune Bitcoin’s blockchain,” in Proc. 2020 IFIP Netw. Conf., pp. 298–306. [52] G. Serugendo, M.-P. Gleizes, and A. Karageorgos, Eds. Natural Computing Series. Cham: Springer Nature Switzerland AG, 2011. [53] C. Badertscher, P. Gaži, A. Kiayias, A. Russell, and V. Zikas, “Consensus redux: Distributed ledgers in the face of adversarial supremacy,” Cryptology ePrint Archive, Rep. 2020/1021, 2020. [Online]. Available: https://eprint.iacr.org/2020/1021 [54] Y. Gilad, R. Hemo, S. Micali, G. Vlachos, and N. Zeldovich, “Algorand: Scaling byzantine agreements for cryptocurrencies,” in Proc. 26th Symp. Operat. Syst. Princ., ACM, 2017, pp. 51–68, doi: 10.1145/3132747.3132757. [55] R. Li, Z. Zhao, X. Xu, F. Ni, and H. Zhang, “The collective advantage for advancing communications and intelligence,” IEEE Wireless Commun., vol. 27, no. 4, pp. 96–102, 2020, doi: 10.1109/MWC.001.1900417. [56] L. Lamport, private communication, May 28, 1987.

Önder Gürcan, Université Paris-Saclay, CEA, List, Palaiseau, F-91120, France. Email: [email protected]. 

©SHUTTERSTOCK.COM/KYRYLO GLIVIN

Detect-Remove-Replace A Robotic Solution That Enables Unmanned Continuous 3D Printing By Hsieh-Yu Li, Yuchen Ma, M. Naufal A. Bin Miswadi, Long Nguyen Nguyen Luu, Liangjing Yang, Shaohui Foong, Gim Song Soh, Espen Sivertsen, and U-Xuan Tan

F

used filament fabrication (FFF) 3D printing is one of the most common additive processes due to its low cost and speed of production. There has been growing interest to have a fleet of such printers, which can be procured at low cost, as they have enormous potential of providing a large volume of continuous production of customized parts on demand. However, the main bottleneck for the FFF printing process is that the part must be manually removed before the next print can commence as the print bed is occupied. Time is also wasted when erroneous print is not stopped. This requirement increases the downtime of the machines significantly, which affects utilization and profit. The process is desired by several industries for continuous printing around the clock. Therefore, this article presents a system with vision-based failure detection and robotic

Digital Object Identifier 10.1109/MRA.2021.3103478 Date of current version: 19 November 2021

1070-9932/22©2022IEEE

manipulation, termed detect-remove-replace (DRR), to address the aforementioned issues. It provides automated monitoring and part extraction for FFF 3D printers while eliminating the need for manual intervention to enable unmanned continuous printing. The experimental results show that DRR is able to detect common defects of the prints (such as spaghetti, detachment, and air printing) with 85.11% accuracy. In addition, the prints are successfully removed 20 times out of 21 tests, and the heatbed steel sheet is robustly replaced for the next prints with a 100% success rate. FFF 3D Printing in Manufacturing 3D printing is an additive manufacturing process where 3D objects and components are built by depositing melted material layer by layer. Among the many 3D printing technologies, FFF is one of the most popular methods due to its low cost and speed of production [1]. In addition, there has been an increasing demand for running a fleet of low-cost FFF printers continuously. A fleet of FFF 3D printers increases JUNE 2022

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

33

the efficiency of the process, providing great advantages for mass production. However, one main issue of continuously printing is that significant downtime is wasted due to two key reasons [1], [2], as shown in Figure 1. First, during the preparation on a new print job or removal of the completed print job, the FFF printers are unutilized, without human interference [2].

The Print needs to be manually removed before the next print can start.

Second, there are various quality problems of the parts that are fabricated using the FFF process, which potentially lead to failure printing [3]. To make matters worse, the machine time is wasted if the defective prints are not stopped. These two issues cause a major hurdle to scale FFF 3D printers. In the current practice, it relies heavily on human intervention to monitor the printing conditions, remove the current print,

Current Practice

Time is wasted if the defective print is not stopped

(a)

Our Proposal (b) Figure 1. The research motivation. A fleet of FFF 3D printers for 24-hr continuous printing provides great advantages. However, in the current practice, to achieve continuous printing is challenging as it requires continuous human intervention for two key reasons. First, the print must be removed before the next print can start. Second, time is wasted if the defective print is not stopped. Hence, in this article, we propose a robotic solution to address these issues.

34

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

JUNE 2022

and prepare for the next print. This becomes challenging for manufacturing as the cost of an idle line is high, along with the manpower-shortage problem. To solve the aforementioned two issues, researchers have proposed two solutions, that is, robotic systems to remove the prints upon completion of the printing [2], [4]–[7] and vision-based failure detection to ensure better quality control [8]–[14]. First, current solutions for part extraction from FFF 3D printers rely on robotic [2], [6], [7] or automated systems [4]. The most straightforward idea is to use a robotic manipulator with a gripper on the end effector to grasp the prints when completed [6], [7]. However, the sizes of the prints that can be removed are limited by the stroke of the gripper. In addition, it is practically challenging for the robot to remove parts that are low in height [1] due to an insufficient gripping area. On the other hand, it has been proposed that a conveyor be installed on the print bed for each printer to remove the print upon completion [4]. Although this promising idea resolves the limitation of the removal of different sizes of the print, it requires customized mechanical designs for each type of printer. This leads to expensive costs as a result of the large-scale setups. Second, several researchers have investigated the area of monitoring in an extrusion-based additive manufacturing process [3], [9]–[14], [21]–[23]. Deep learning (DL)based vision detection is one of the methods, and it has become popular in recent years due to its accuracy. One

example is The Spaghetti Detective (TSD) [10], a You Only Look Once (YOLO)-based algorithm [15] with an open source code used to detect a common defect, called spaghetti. As shown in Figure  2(a), the failures happen when the print pops off the print surface in midprint, and a vast pile of spaghetti is produced over the remaining print [10]. Zhang et al. [11] and Kim et al. [22] have developed a customized convolutional neural network to detect material-extrusion failures. Although a DL-based algorithm provides decent accuracy on detecting spaghetti, it fails to detect other types of failures or any failures that can be recognized with only a sequence of images. These failures require the context of the printing process, e.g., previous images and current expected work piece states. The two common examples without spaghetti are shown in Figure  2(b) and (c), namely, detachment (the print “detaches” from where the filament is extruded) and air printing (no filament is extruded due to the nozzle clogging or running out of filament). To address the issue, Lyngby et al. [12], Straub [13], and Nuchitprasitchai et al. [14] introduce various vision algorithms based on one or two cameras to detect failures like detachment and air printing. The proposed algorithms utilize the comparison between the current and previous printing state, or between the current and desired image to check the conditions of the prints. Although they achieve a great detection rate on air printing, in contrast to TSD, spaghetti is not considered in their reports. In addition, one main downside

Spaghetti

Detachment

Air Printing (Nozzle Clogging/Running Out of Filament)

(a)

(b)

(c)

Figure 2. The three common types of failures in FFF 3D printing. (a) Spaghetti is where a pile of filament comes out without forming a structure, (b) detachment occurs when the print is detached from where the filament is extruded, and (c) air printing is the result of the filament is not being extruded.

JUNE 2022

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

35

of their methods is that an extra customized hardware setup is needed (a moving printing platform [3], rotational camera [9], and multiple cameras [13], [14]). Some setups [9] also obstruct the path of the robotic manipulation. Hence, those methods Among the many 3D are not suitable to be implemented for the printing technologies, FFF objective of having a fleet of low-cost printers and is one of the most popular not suitable for robotic methods due to its low cost manipulation. It seems that the requirement lies on an algorithm 1) for and speed of production. more various types of failure detection, 2) with a low-cost hardware setup, and 3) where the setup for failure detection does not obstruct the path for the robot to replace the prints. Contribution Based on the identified needs, we therefore propose a workflow, termed DRR, for automated monitoring and part extraction for FFF 3D printers. This design aims to achieve unmanned continuous printing for a fleet of low-cost 3D

printers. The proposed system utilizes a hybrid image comparison (IC) and DL method to detect the failure of the printing. In addition, a robotic system is developed to remove the prints upon completion of the printing or detection of failure, followed by replacing the heatbed steel sheet for the next print. To the best of our knowledge, the proposed workflow is the first integration system based on vision and the robot for automated monitoring and part extraction for FFF 3D printers. The contribution is twofold. First, the integrated framework reduces the need for human intervention, hence providing the capability for unmanned continuous printing. Unlike existing research in robotic manipulation where nondefective prints are the main focus, both nondefective and defective prints are considered in this work, which is important for continuous printing. Second, the DRR algorithm addresses the problem of existing research where only specific sizes of prints can be replaced and only the spaghetti type of failure detection can be detected. The proposed grasping manipulation aims to provide a robust removal of the prints with fewer mechanical setups on each printer, and the proposed vision algorithm aims to achieve the detection of more types of common failures. These two aspects cover the identified needs.

(c)

(b)

(a)

(d)

(e)

Figure 3. The system overview (hardware). Three Prusa I3 MK3S are set up as the FFF printer. Each printer is installed with mechanical guides, a Raspberry Pi, and an RPi camera. A UR10e is the manipulator, and a removal area is developed for the robot to remove the prints. (a) The proposed robotic system with a fleet of 3D printers. (b) A 3D printer, (c) gripper, (d) RPi camera, and (e) removal area (for bending and scraping the sheet).

36

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

JUNE 2022

The Proposed Robotic System System Overview As depicted in Figure 3(a)–(e), the developed robotic system consists of three parts: the robotic system, three FFF 3D printers, and a removal area. First, a UR10e (Universal Robots, Denmark) is used as our robotic manipulator, with a two-jaw parallel gripper (GEP2016IO-05-A, Zimmer Group, Germany). Second, three FFF 3D printers (Original Prusa I3 MK3S, Prusa Research, Czech) are installed, with each having one Raspberry Pi (RPi) 4 Model B. Each RPi is connected with a 8MP RPi camera (Camera Module v2). In each printer, there is a flexible and removable heatbed steel sheet that is magnetically connected to the print bed. The Raspberry Pi is communicated with the host controller (an Ubuntu 16.04 computer) via Wi-Fi to enable remote access. OctoPrint (https:// octoprint.org/; https://github.com/OctoPrint/OctoPrint) is used in each RPi to control the printers with Octoclient (https://github.com/hroncok/octoclient), while Octolapse (https://github.com/FormerLurker/Octolapse) is utilized to make stabilized, time-lapse images of our print. In addition, two customized mechanical guides are mounted with each printer to ensure the correct replacement of the heatbed steel sheet. Third, a removal area is designed to remove the prints from the heatbed steel sheet. More details will be discussed in future sections of this article. DDR The DRR algorithm comprises three components: 1) failure detection, 2) robotic manipulation for removal of the print, and 3) robust replacement of the heatbed steel sheet. The entire flow is presented in Figure 4(a)–(c). Detect: Failure Detection The hybrid IC and DL vision (HIDV) algorithm is proposed to detect the failures. In this article, we focus on the detection on three common types of failures, namely, spaghetti, detachment, and air printing. More specifically, an IC [16], [17] is employed to handle detachment and air printing. In addition to an IC, the YOLO-based TSD [10], [15] is used specifically for spaghetti. The pseudocodes of the HIDV algorithm are shown in “Algorithm 1: Hybrid IC and DL Vision Algorithm.” Image Comparison to Detect Detachment and Air Printing First, the proposed IC algorithm is inspired by [16] and [17]. The flow of the IC algorithm is shown in Figure 5. During the printing, an image frame is taken for each layer of the print. In each step of the calculation in the IC algorithm, there are four images being considered. The four images are I BG, I C, I P, and I 5L, which denote the background image, current image, image from the previous layer, and image from five layers ago, respectively. At the begging of the print, an image is taken as the background image. In Figure 5, the detachment is illustrated as the example for the current image I C . I P is the

image in the previous layer of I C, and I 5L is the image from five layers ago of I C . There are four procedures in each step of calculation in an IC, namely, gray scale, absolute difference, threshold, and normalized root mean-square error (NRMSE). More specifically, the first procedure converts color images into gray scale. In the second procedure, given two images, absolute difference aims to detect the motion of the objects [19]. Assume I k to be the value of the kth frame in the image sequences, and the absolute differential image is defined as [19]

I d (k, k - 1) = ; I k - I k - 1 ; .(1)

In this example, assume the current frame is the kth frame, hence, I BG, I C, I P, and I 5L represent I 0, I k, I k - 1, and I k - 5, respectively. In the second procedure, the absolute difference is employed to obtain the difference among the three images (i.e., I C, I P, and I 5L) and the background image (I BG), which are (HTML translation failed) and I d (5L, BG) . The three absolute differential images are binarized, using a binary threshold in the third procedure, and got binary images I t (C, BG), I t (P, BG), and I t (5L, BG) . Finally, NRMSE is used to compare the structures of two chosen images. Assuming the two images are k and k - 1, NRMSE (k, k - 1) is defined [16], [20] as

RMSE (k, k - 1) =

1 N

N

/ (I k (i ) - I k -1(i ))2 (2)

i =1

Algorithm 1: Hybrid IC and DL Vision Algorithm Given k as number of layers with image I k Obtain the image in the current layer I C ! I k if k = 0 then  Register I 0 as the background image I BG , I BG ! I 0 else if k # 5 then   Save images I k else   Save images I k  Obtain I 5L , I P and I C  Calculate U (C, 5L) and U (C, P ) , then check the printing conditions using IC [16], [17]  if U (C, 5L) 2 U (5L, max) and U (C, P ) 2 U (P, max) then    The Detachment occurs  else if U (C, 5L) 1 U (5L, min) and U (C, P ) 1 U (P, min) then    The Air-Printing occurs   else    Check the printing conditions using DL    Import TSD [10], [15] model M TSD   if detection 2 0 using TTSD then    The Spaghetti occurs   end if   end if end if if Spaghetti, Detachment or Air-Printing occur then   Stop printing.   Activate robot to execute Remove and Replace else   Continue printing. end if Execute the next layer if no errors occur.

JUNE 2022

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

37



U (k, k - 1) / NRMSE (k, k - 1) = RMSE (k, k - 1)

N , (3) ; Ik ;

where N is the number of pixels in the image. Hence, U (C, P) is considered the difference between the current image I C and previous image I P, while U (C, 5L) represents the difference between I C and I P .

Hybrid IC/DL Vision Algorithm

1)

As presented in Figure 5, the detachment happens in the current layer. The detachment areas cause a significant difference in I d (C, BG), compared to I d (P, BG) and I d (5L, BG) . It also leads to larger values on U (C, 5L) and U (C, P) . On the contrary, if both U (C, 5L) and U (C, P) are small values, this indicates only nuances of the changing between the current and previous layers, and the current layer and five layers ago. The result

(a)

2)

3)

4)

(b)

2)

1)

(c) Figure 4. The proposed workflow of DDR. (a) Detect: A hybrid IC [16], [17] and DL [10], [15] vision algorithm is used for failure detection. (b) Remove: The robotic manipulation is developed to remove the print by 1) removing the heatbed steel sheet for 2) bending, 3) flipping, and 4) scraping. (c) Replace: The force control is employed to 1) robustly replace the heatbed steel sheet for 2) the next print.

38

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

JUNE 2022

then suggests that the layers might be not increasing due to nozzle clogging or a lack of filament (air printing). Therefore, the effect of U (C, 5L) and U (C, P) can be summarized as in Table 1, where U 5L, max, U 5L, min, U P, max, and U P, min are the threshold for the max and min values of the image five layers ago and the previous image, respectively. YOLO-Based Model to Detect Spaghetti After the IC is obtained, then DL is then used to check for the spaghetti. In this article, TSD [10], a pretrained YOLObased model, M TSD , is imported. The M TSD model processes each image and returns all the spaghetti patterns that it detects with confidence scores. A threshold in confidence scores is set to filter out noises, and the current print is classified as “failed” if the number of defective patterns detected with a confidence above the threshold is greater than zero. As shown in the pseudocodes, if no errors (spaghetti, detachment, air printing) are detected, the printing continues and the next layer process is executed. Conversely, if any defect is detected, the printing is forced to stop, followed by the robot executing Remove and Replace, which will be discussed in the next section.

Background Image, IBG

Remove: Robotic Manipulation for Removal of the Print Once the failure detection occurs or the print is complete, the next step is to use the robot to remove the current print. Usually, as in the case of manual removal, humans use their hands to remove the print, or tools (such as scrappers) for the prints with strong adhesive forces on the heatbed steel sheet. In this way, different sizes of the prints can be easily removed. However, imitating such human skills to use the tool poses a challenge on robotic manipulation. This is because many factors are then required to be considered for

Table 1. Threshold of the image comparison for each of the failures. Detection

Results

Detachment

U (C, 5L) 2 U 5L, max and U (C, P ) 2 U P, max

Air printing

U (C, 5L) 1 U 5L, min and U (C, P ) 1 U P, min

Others or no failures

Others

Five Layers Ago, I5L

Previous Layer, IP

Current Layer, IC

4) NRMSE

3) Threshold

2) Absolute Difference

1) Gray Scale

Detachment

IBG

I5L

IP

IC

Id(5L, BG )

Id(P, BG )

Id(C, BG )

It(5L, BG )

It(P, BG )

It(C, BG )

(C, P)

(C, 5L)

Figure 5. The IC algorithm, one of the algorithms in Detect (with detachment as the example). Assuming detachment occurs in the current layer, the images from the previous layer, five layers ago, and the background images are compared with the absolute difference, threshold, and normalized root mean-square error (NRMSE), respectively, to find the difference. The last two results, U (C, 5L) and U (C, P ) , classify the conditions of the printing as detachment, air printing, or others.

JUNE 2022

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

39

manipulation, such as grasping points and orientation on the tools, interaction forces on heatbed steel sheet, or scraping direction on the prints. Therefore, to overcome limitations of the print size and manipulation challenges, we propose a different way for the removal of the prints based on the features of the flexible heatbed steel sheet. This novel robotic removal manipulation step includes four substeps, as shown in Figure 6(a). First, the print is taken out with the robot grasping on the heatbed steel sheet. Second, the robot inserts the heatbed steel sheet to a “jig” on the removal area, followed by a bending of the steel sheet. This step is critical to remove the print as the bending of the steel sheet separates the adhesion between the print and the heatbed steel sheet. It allows the prints to easily drop out of the sheet after the sheet is flipped in the next substep. The third step flips the heatbed steel sheet, followed by a scraping of the rest of the prints (such as the skirt or other residuals) off in the last step. To scrape the print off, the robot presses the heatbed steel sheet against another jig with strong force. To guarantee the print is completely removed, the robot repeatably scrapes the heatbed steel sheet against the jig backward and forward.

Replace: Robust Replacement With Force Control Upon the removal of the current print, a robotic manipulation is performed for the replacement for the next print. The most challenging part in this step is how to ensure the robustness of the print bed placement. This is because the sheet is typically displaced by the gripper during the removal process as high interaction forces are involved. This results in a different grasping point from the moment that the robot takes the heatbed steel sheet in the remove step and the moment that the robot places it back in the Replace step. It poses another challenge for robotic manipulation to accurately replace the heatbed steel sheet back on the printer bed. Although a vision system is a promising solution to close the loop and remove the error, the cost of the setup will increase. In addition, given the limited space on each printer and the gripper, installing extra sensors becomes troublesome and might disrupt the path of the manipulation. Hence, we propose force control with mechanical guides and a stopper to address the issue in this step. The flow of this robust replacement is displayed in Figure 6(b) and comprises three substeps. First, the robot grasps the heatbed steel sheet and places it on the assigned location with the front of the sheet in contact with the print bed. Second, three mechanical parts

1) Remove the Heatbed Steel Sheet

2) Bending Process

3) Flip the Heatbed Steel Sheet

4) Scrape the Print Off

Jig for Bending

Jig for Removal

(a)

Stopper

Mechanical Guides

Push Mechanical Structure 1) Place the Heatbed Steel Sheet

2) Push the Heatbed Steel Sheet (Force Control)

3) Finish the Replacement

(b) Figure 6. Remove and Replace. (a) This step removes the current print, with the robot grasping the heatbed steel sheet to scrape the print off on the removal area. It consists of four substeps, namely, 1) taking out, 2) bending, and 3) flipping the heatbed steel sheet in the first three substeps, followed by 4) removing the print in the last substep. (b) In this step, the robot places the sheet back on the print bed. To overcome the challenges of where the sheet is not accurately placed in the same position after multiple attempts, we develop a specific system where mechanical guides and a stopper are built to correct the path of the placement of the heatbed steel sheet. In addition, force control is employed to guarantee a stable interaction.

40

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

JUNE 2022

Table 2. The success rates in experiment 1: Failure detection (Detect). Method

Nondefective Prints

Spaghetti

Detachment

Air Printing

Overall Failure Detection

Proposed method

100% (8/8)

100% (13/13)

88.24% (15/17)

70.59% (12/17)

85.11% (40/47)

TSD

100% (8/8)

100% (13/13)

64.71% (11/17)

0% (0/17)

51.06% (24/47)

are designed to tackle the problem of the inaccuracy of the placement. The first is the push mechanical structure on the gripper where the finger of the printer is designed as an L shape from the side view [as shown in Figure 6(b)]. 1) The robot pushes the sheet using the push mechanical structure with the mechanical guides and the stopper on the printer to guide the path of the sheet that the robot is pushing. In addition, the motors of the printer are activated to provide the holding force when the sheet is pushed. To guarantee a stable interaction, a simple force control is employed as follows:

Vd = K f (Fd - F f bk), (4)

where Vd is the desired speed of the robot, K f is the force control gain, and Fd, and F f bk are the robot’s desired force and force feedback, respectively. A stable motion can be achieved using (4) when the robot pushes, with a strong force, the sheet against the stopper with the activated printer motors. Finally, the robot opens the gripper to place the sheet on the print bed and completes the replacement. Experiments Three experiments are conducted. Failure detection (Detect) is first validated, followed by the validation of robotic manipulation for Remove and Replace. Finally, the entire flow of the DRR workflow is demonstrated with three printers and one robot. The demonstration of capability can also be found in the accompanying video (available in https://doi.org/10.1109/ MRA.2021.3103478). Experiment 1: Validation of Failure Detection (Detect) The first experiment is conducted to validate the proposed HIDV algorithm for failure detection. The experiment includes three sets of experiments, namely, nondefective prints, spaghetti, and detachment and air printing. The parameters are as follows: U 5L, max = 0.50, U 5L, min = 0.38, U P, max = 0.60, and     U P, min = 0.30. The proposed method is compared with the traditional YOLO-based detection [10] in each set. As our focus in this article is to develop a robotic solution to enable continuous printing for a fleet of low-cost printers, this comparison set has the following conditions: 1) minimal hardware setup is required and 2) the setup for failure detection does not block the path for the manipulator to replace the prints.

Hence, some existing methods are not suitable for this purpose, such as extra customized hardware [3], [11] or an obstruction in the path of robotic manipulation [8], [9]. In addition, to test the versatile and various prints, the prints with different colors, shapes, and sizes are chosen. The results of the success rates are listed in Table  3, and the examples of detection using the proposed method are presented in Figure 7. The first subset utilizes vision-detection algorithms to observe the printing that has no failures (nondefective prints). The result shows t hat b ot h met ho ds achieve a 100% success This work is the first rate, meaning no failures are detected. The second solution for continuous subset is a common defect in FFF 3D printing: printing on multiple lowspaghetti. The spaghetti effects are created from cost FFF 3D printers, where either the purposely poorly designed files both automated monitoring where defects naturally (such as the prints withand part extraction are out supports), or by manual operation (inintegrated to eliminate crease the temperatures, push the print, and so human intervention. on), occur. As shown in Table  2, both methods achieve the same success rate. This is not surprising as the proposed method is developed based on the traditional method. However, the results from the third and fourth subset are significantly different between the two methods. The other two common types of failures, that is, detachment and air printing, are conducted in these two subsets of experiments. Both defects are manually created (detachment is generated, with the print being stopped and the human removing the print, while air printing is caused by reducing the nozzle temperatures). As presented in Table  2, TSD is unable to detect detachment and air printing as well, where it is

Table 3. The success rate in experiment 2: Robotic manipulation (Remove and Replace). Method

Remove

Replace

Proposed method

95.24% (20/21)

100% (20/20)

JUNE 2022

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

41

(a)

(b)

(c)

(d)

Figure 7. The results of experiment 1 in which the HIDV is evaluated. First, the proposed vision algorithm guarantees that (a) nondefective prints are not being stopped. Second, the defective cases where (b) spaghetti, (c) detachment, and (d) air printing are successfully detected by HIDV are demonstrated.

42

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

JUNE 2022

especially true for air printing. This is not surprising as it is designed to detect spaghetti. Conversely, the proposed method is able to achieve detection 27 times out of 34 tests. The total success rate on three types of failures is 85.11%, which demonstrates that the proposed HIDV addresses the issue, where the traditional method can only detect the failure based on analysis of a single image. One thing to take note of is that the developed HIDV for Detect provides flexibility according to the needs of the users as it relies on some tradeoff among several factors. One of the tradeoffs is the way to mount the camera. The camera in this article is mounted in a fixed position, which benefits detection on more types of failures, but the vision algorithm also becomes sensitive to the motor movement. Another mounting is where the camera relocates along with the motor that moves vertically, which reduces the sensitivity in camera movement as it only focuses on the nozzle area where the filament is extruded. However, it also limits the same type of detection as the camera might not see the print bed when the motor goes higher. Experiment 2: Validation of Robotic Manipulation (Remove and Replace) The goal of the second experiment is to validate the proposed Remove and Replace using robotic manipulation. The robustness of these two steps are important for continuous printing as it requires human intervention if the robot fails to replace the heatbed steel sheet. In this experiment, the printings with different sizes and dimensions are tested. To verify the capability of robotic manipulation on various types of prints, both nondefective and defective prints (spaghetti and detachment) are carried out. The defective prints are not stopped by our vision algorithm in this experiment so that there are more spaghetti or scattered detachment defects,

which are more challenging for removal. Upon completion, the robot grasps the heatbed steel sheet with the prints and executes the remove step, followed by a Replace step for the next print with 8  N as the desired force (the built-in force control in the UR10e). In addition, the cases where the adhesive force between the prints and sheet is stronger than the specification of the gripper are not considered in this experiment.

The results are shown in Table 2 and Figure 8. In the remove step, the nondefective prints are successfully removed 13 times out of 13 tests, with the size ranging from a small point (5 mm in width and height) to a large print that covers nearly the entire heatbed steel sheet. In addition, the removal of the defective prints is difficult as the prints are scattered all over the place, and the robot is still able to successfully remove defective prints seven times out of eight

Small Sizes

Medium Sizes

a small point

with supports

Large Sizes

half of the print bed

long and thin

tall

entire sheet (a) Failed Manipulation The prints drop out of the sheet.

(b)

Figure 8. The results of experiment 2 in which the proposed manipulation method for part extraction is evaluated. The proposed method achieves a 95.24% success rate for removing the current prints and replacing the heatbed steel sheet for the next print. The test sets include the print with and without failures.

JUNE 2022

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

43

tests. The failed manipulation case occurs because the prints or the spaghetti drop out of the sheet if there is not enough adhesive force. This can be prevented in the entire flow, as the vision algorithm will stop the printing before more spaghetti or detachment occur. The results show the robustness of our method to remove the prints [95.24% (21/22) success rate in total]. The results also demonstrate the capability of robotic manipulation on various types of prints. Finally, the Replace step is executed after remove. The failed case on remove is not considered in Replace as the heatbed steel sheet has dropped and the robot is not able to continue the manipulation. As presented in Table 2, across the 20 tests, all of the heatbed steel sheets are successfully placed back on the print bed. This suggests that the developed mechanical guides and stopper with the proposed force control could benefit from the overall robustness and stability of preparing for the print. Although the proposed method requires one more step (Replace) to place the sheet back, with a 100% success rate in the Replace step, the overall success rate in the two steps (95.24%) shows the robustness of Removal and Replace. Of note is that the manipulation time is 3 min 21 s on average (as there are three printers with three paths) between the end of the print and the start of the adjacent print. In addition, as depicted in Figure 9, grasping directly on the prints is not suggested. This is because the prints can be of arbitrary shape, which is extremely difficult for the gripper to grasp directly (e.g., gripper size and detecting the grasping point). Hence, this method is avoided in our proposal.

detect the failures. In addition, the robot successfully removes the current print and places the heatbed steel sheet back for the next print. The entire process is conducted without human intervention, showing that the proposed solution enables continuous 3D printing, which demonstrates our main contribution. One thing to consider is that, in this article, one bin is used to collect the complete or defective prints. It would be useful for future work to place two bins, one for successful prints and another for defective prints. In addition, the limitation of the proposed method also suggests directions for future investigations of failure detection. The proposed HIDV provided relies on sudden image differences or spaghetti, it lacks the capability to detect slow-varying defects, such as warpage. Hence, it would be interesting to combine other methods [18] that analyze the current and expected printings from models of the files to address the limitation.

Conclusions In this article, a robotic solution was proposed for unmanned continuous FFF 3D printing. A DRR workflow was developed to automatically monitor the conditions of the FFF 3D printing while executing the part extraction when the print has completed or failed. This study also addressed the bottleneck on existing research where limited types of failures can be detected and only certain sizes of prints can be removed. The experiments were conducted to validate the idea. First, the proposed hybrid IC/DL vision algorithm achieved an 85.11% accuracy of failure detection on common defects (spaghetti, detachment, and air printing). Second, DRR provided robust robotic manipulation on part extraction for the current print (20 times out of 21 tests) and replacement of the heatbed steel sheet for the next print (100%). Experiment 3: Validation of the Entire Workflow The main contribution is twofold. First, to the best of our In the final validation, we conduct an experiment where the knowledge, this work is the first solution for continuous printentire DRR workflow is demonstrated, using three printers ing on multiple low-cost FFF 3D printers, where both automatand the robot. The goal is to test the robustness of the contin- ed monitoring and part extraction are integrated to eliminate uous printing and detection and to verify the flow of the sys- human intervention. Second, the DRR algorithm achieves sattem integration on multiple printers. The results can be isfactory results on the accuracy of different types of failure found in the video. During the 11 printings, HIDV is able to detection and the success rate of different sizes of part extraction. With the proposed workflow, we hope to advance the level of autonomy in additive manufacturing to provide Successful Failed Manipulation (the Sizes of Prints are Larger Than better efficiency by reducing the need Manipulation the Stroke of the Gripper) for human operations.

(a)

(b)

(c)

(d)

Figure 9. The traditional way of removing the prints. The sizes of the prints are limited due to the stroke of the gripper. In addition, the orientation and position of the prints need to be considered, which increases the challenges to the removal. (a) A successful manipulation and (b)–(d) unsuccessful manipulations where the sizes of the prints are larger than the stroke of the gripper.

44

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

JUNE 2022

Acknowledgments This research is supported by NAMIC Singapore and funded by the National Research Foundation Singapore under its Innovation Cluster Program. The corresponding author is Hsieh-Yu Li. This article has supplementary downloadable material available at https:// doi.org/10.1109/MRA.2021.3103478, provided by the authors.

References [1] M. N. A. B Miswadi, “Automated monitoring and part extraction for fused filament fabrication (FFF) printers,” Master’s thesis, Singapore Univ. of Technology and Design, 2019. [2] R. V. Aroca, C. E. H. Ventura, I. De Mello, and T. F. P. A. T. Pazelli, “Sequential additive manufacturing: Automatic manipulation of 3D printed parts,” Rapid Prototyping J., vol. 23, no. 4, pp. 653–659, 2017. doi: 10.1108/RPJ-02-2016-0029. [3] Y. Tlegenov, G. S. Hong, and W. F. Lu, “Nozzle condition monitoring in 3D printing,” Robotics Comput.-Integrated Manuf., vol. 54, pp. 45–55, Dec. 2018. doi: 10.1016/j.rcim.2018.05.010. [4] N. B. Pettis, “Automated 3D build processes”, U.S. Patent 8 668 859 B2, Mar. 11, 2014. [5] O. Brockmeier, “Automated loading and unloading of the Stratasys FDM 1600 Rapid Prototyping System,” M. Sc thesis, Virginia Polytechnic Inst. and State Univ., 2000. [6] B. Benchoff, “Automating 3D Printers with Robots tend.ai,” June 2016. Accessed: Oct. 13, 2020. [Online]. Available: https://hackaday .com/2016/06/27/automating-3d-printers-with-robots/ [7] “Voodoo Manufacturing triples 3D printing production with collaborative robots,” Voodoo Manufacturing, 2017. [Online] Available: https://www.universal-robots.com/case-stories/voodoo-manufacturing/ [8] D. Ugandhar and S. Chang, “Automated process monitoring in 3D printing using supervised machine learning,” Procedia Manuf., vol. 26, pp. 865–870, Aug. 2018. doi: 10.1016/j.promfg.2018.07.111. [9] Y. Wang et al., “A CNN-based adaptive surface monitoring system for fused deposition modeling,” IEEE/ASME Trans. Mechatron., vol. 25, no. 5, 2020, pp. 2287–2296. doi: 10.1109/TMECH.2020.2996223. [10] K. Jiang, “The Spaghetti Detective,” GitHub, San Francisco, 2012. Accessed: Oct. 13, 2020. [Online]. Available: https://github.com/TheSpa ghettiDetective/TheSpaghettiDetective [11] Z. Zhang, I. Fidan, and M. Allen, “Detection of material extrusion in-process failures via deep learning,” Inventions, vol. 5, no. 3, p. 25, July 2020. doi: 10.3390/inventions5030025. [12] R. A. Lyngby et al., “In-line 3D print failure detection using computer vision,” in Proc. Dimensional Accuracy Surface Finish Additive Manuf., Oct. 2017. [13] J. Straub, “Initial work on the characterization of additive manufacturing (3d printing) using software image analysis,” Machines, vol. 3, no. 2, pp. 55–71, 2015. doi: 10.3390/machines3020055. [14] S. Nuchitprasitchai, M. C. Roggemann, and J. M. Pearce, “Three hundred and sixty degree real-time monitoring of 3-d printing using computer analysis of two camera views,” J. Manufact. Mater. Process., vol. 1, no. 1, 2017. [Online]. Available: https://www.mdpi.com/2504-4494/1/1, doi: 10.3390/jmmp1010002. [15] J. Redmon, S. Divvala, R. Girshick and A. Farhadi, “You only look once: Unified, real-time object detection,” in Proc. IEEE Conf. Comput. Vision Pattern Recogn. (CVPR), Las Vegas, NV, 2016, pp. 779–788. doi: 10.1109/CVPR.2016.91. [16] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Trans. Image Process., vol. 13, no. 4, pp. 600–612, Apr. 2004. doi: 10.1109/ TIP.2003.819861. [17] B. Withers, “3DPrintSaviour,” GitHub, San Francisco, 2018. Accessed: Oct. 13, 2020. [Online]. Available: https://github.com/Manicben/ 3DPrintSaviour

[18] F. Baumann and D. Roller, “Vision based error detection for 3D printing processes,” MATEC Web Conf., vol. 59, p. 06003, 2016. doi: 10.1051/matecconf/20165906003. [19] N. Singla, “Motion detection based on frame difference method,” Int. J. Inform. Comput. Technol., vol. 4, no. 15, pp. 1559–1565, 2014. [20] Pedregosa et al., “Scikit-learn: Machine Learning in Python,” J. Mach. Learn. Res., vol. 12, pp. 2825–2830, Feb. 2011. [21] K. Okarma and J. Fastowicz, “Improved quality assessment of colour surfaces for additive manufacturing based on image entropy,” Pattern Anal. Appl., vol. 23, no. 3, pp. 1035–1047, 2020. doi: 10.1007/ s10044-020-00865-w. [22] H. Kim, H. Lee, J.-S. Kim, and S.-H. Ahn, “Image-based failure detection for material extrusion process using a convolutional neural network,” Int. J. Adv. Manuf. Technol., vol. 111, nos. 5–6, pp. 1291–1302, 2020. doi: 10.1007/s00170-020-06201-0. [23] O. Holzmond and X. Li, “In situ real time defect detection of 3D printed parts,” Additive Manufacturing, vol. 17, pp. 135–142, Oct. 2017. doi: 10.1016/j.addma.2017.08.003.

Hsieh-Yu Li, Pillar of Engineering Product Development, Singapore University of Technology and Design, 487372, Singapore. Email: [email protected]. Yuchen Ma, Pillar of Engineering Product Development, Singapore University of Technology and Design, Singapore, 487372, Singapore. Email: [email protected]. M. Naufal A. Bin Miswadi, Group Enterprise, Singapore Telecommunications Limited, 239732, Singapore. Email: naufal. [email protected]. Long Nguyen Nguyen Luu, Pillar of Engineering Product Development, Singapore University of Technology and Design, 487372, Singapore. Email: [email protected]. edu.sg. Liangjing Yang, Zhejiang University/University of Illinois at Urbana-Champaign Institute, Haining, 310027, China. Email: [email protected]. Shaohui Foong, Pillar of Engineering Product Development, Singapore University of Technology and Design, 487372, Singapore. Email: [email protected]. Gim Song Soh, Pillar of Engineering Product Development, Singapore University of Technology and Design, 487372, Singapore. Email: [email protected]. Espen Sivertsen, Ivaldi Group, San Leandro, California, 94577, USA. Email: [email protected]. U-Xuan Tan, Pillar of Engineering Product Development, Singapore University of Technology and Design, 487372, Singapore. Email: [email protected].  JUNE 2022

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

45

©SHUTTERSTOCK.COM/FLOWGRAPH

TUTORIAL

The Franka Emika Robot A Reference Platform for Robotics Research and Education By Sami Haddadin, Sven Parusel, Lars Johannsmeier, Saskia Golz, Simon Gabl, Florian Walch, Mohamadreza Sabaghian, Christoph Jähne, Lukas Hausperger, and Simon Haddadin

T

he importance of robots for industry, research, education, and society as a whole is steadily increasing as reflected by the number of available systems and installed robots, not only in industry but also in the public sector and households. Software-only robotics researchers usually rely on commercially available robots which, in the case of manipulators, are primarily designed for industrial purposes and are often far from their needs. This article is a hands-on tutorial on the Franka Emika robot, the first series of industrial artificial intelligence (AI)-ready tactile robot platforms. Beyond industrial use, the systems can be seamlessly expanded to fulfill the demands of research and

Digital Object Identifier 10.1109/MRA.2021.3138382 Date of current version: 25 February 2022

46

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

JUNE 2022

education across all robotics and AI disciplines. To satisfy the needs of such a wide variety of fields, it provides three different interfaces: Desk, a high-level app-based user interface for easy and fast task programming; Robot Integrated Development Environment (RIDE), a command-based programming environment used to create high-performance robot skills that enables programming custom apps and integrating external sensors; and the Franka control interface (FCI), a 1-kHz low-level torque and position control interface that exploits the alsoavailable Langrangian dynamics robot model. We take a close look at implementations with all interfaces, ranging from simple solutions, apps, and controllers to robotlearning examples illustrating how to exploit all the advantages of this platform in ongoing robotics research and education. 1070-9932/22©2022IEEE

Introduction From a researcher’s point of view, the influence of a Robotics research involves the development of both new specific platform and its impact on his/her work cannot hardware and software. With the increasing popularity of be overstated. At first glance, its application programAI, software-based robotics research is currently boom- ming interface (API) dictates the necessary implemening. As a software-only branch of development, it must tation effort, while its hardware and low-level interface rely on readily available high-performance robotic plat- determines the robot-specific control performance of forms, which, in the case of manipulation, are mostly new approaches. However, the plurality and flexibility limited to industrial robotic arms. Today, manufacturing of interfaces strongly conditions the success of multiis the most relevant commercially viable use case for disciplinary works and joint projects. The level of adopautomated manipulation. It drives and shapes the devel- tion by the community and ease of reproducibility of opment of new robot arms and consequently, the pro- competing methods also defines the quality of potential gramming interfaces that these robots provide are evaluations, objective comparisons and available open strongly oriented toward constrained industrial solutions. source code. For dis­sem­ination and outreach activities, To work with these robots, research labs must overtake ease of use and interactivity are key; but also, an overmany difficulties and invest a significant amount of work arching platform that is likewise designed for educato first set them up and then reach the desired signals or tional purposes boosts and deepens the dissemination high-level commands that their specific application of research results into the general public. With this domain requires. In addition, the diversity of applications holistic researcher-centric perspective in mind (see that a research robot should cover is very broad. Robotics Figure 1) for the development of the Franka Emika roencompasses many fields of computer science and engi- bot, we identified the following key goals. Note that neering, including signal processing, control, planning, these goals are deliberately formulated from a rather computer vision, machine learning (ML), or neighboring high-level perspective and serve primarily as motivafields such as cognitive and social neuroscience. The tion for the presented platform. requirements of researchers in all these areas differ sig- ●● Multidisciplinary accessibility: Researchers should be able to nificantly. For instance, from a control engineer perspeceasily access the signals, sensor measurements, and comtive, the ideal platform offers 1-kHz-level joint-torque mands that their applications require. Control researchers interface that exploits an energy-consistent Lagrangian require low-level access to the robot, e.g., direct jointdynamics robot model; it delivers industrial grade pertorque control over a sufficiently fast real-time connection formance, with position repeatability of less than 0.1 mm (1 kHz), while psychologists do not require the programand has accurate force/torque (F/T) sensing capabilities ming of custom low-level controllers from scratch for their at the end effector. experiments. For them, a higher-level interface is signifiIn contrast, psychologists studying human–robot cantly more effective. interaction focus on human behavioral aspects and require an effective way to program higher-level robot Research/Education behaviors, such as possibly compliant, goal-oriented robot motions. Control theory Human–Robot Interaction This obvious focus gap between disFCI ciplines is one of the most challengSignal Processing ing issues for robotics research Motion Planning platforms. Despite some progress in recent years where some available Desk Optimization Theory manipulators have started including Neuroscience a research interface into their prodRIDE ucts (either high or very low level), Ethics Computer Vision these do not fulfill the needs of such an heterogeneous variety of fields. ML In fact, there is possibly no unique Cognitive and Behavioral interface for the whole application Sciences spectrum of robotics research. It thus becomes apparent that robot manufacturers have not previously Implementation Evaluation Dissemination considered robotics research and education in an integral use case Figure 1. The holistic approach of the Franka Emika robot, with three different during the conception of their prod- programming interfaces that satisfy the needs of most robotics research and education fields. ucts at scale. JUNE 2022

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

47

AI-ready: Robot systems are the embodiment of AI. An AIready reference platform should provide easy access to all sensor data at high frequency, and fast control with minimal latency. It must also excel at interaction control to enable safe environment exploration, a key requirement for robot-learning approaches. It should also support the development of AI solutions with AI libraries provided in house for all robotics disciplines. ●● Interaction control: A stable and sensitive physical interaction with the environment is the defining characteristic of manipulation. A robot that excels at controlling it, capable of accurately adapting its rendered compliance and regulating its applied force, is the only way to advance the state of the art. ●● Applications: A reference platform for manipulation must be capable of performing the widest range of manipulation tasks. It must contain high-performance sensors, actuators, and control algorithms that enable all possible behaviors encompassed in robotics research. ●● Easy setup: It requires some degree of expert knowledge to properly set up an industrial robotic platform. A reference platform for research, and especially for educational purposes, should be easily set up in a matter of minutes by every layperson. ●● Sharing and community: A big fraction of the development time of any new method in robotics research is given by its software integration, especially for a new platform. An even bigger fraction consists of reproducing research from other researchers, which is unfortunately often not even possible. A robot for researchers should encourage the community to have an in-house curated platform for technical discussion and code sharing. ●● Target audience: The target audience of traditional robotic systems is usually rather narrow and limited to industrial engineers or robotics researchers, hindering the impact of their work. A research platform should be as close as possible to the general public and connected with all phases of education; it should offer accessible interfaces and interaction modes for everyone to be the embodiment that prepares not only future engineers but also an open door into robotics research for society as a whole. ●● Affordability and availability: The impact and potential reach of a platform is first affected by the quality of its features but mostly defined by its user base. Most industrial robots still require significant financial investments, which prevents them from being available to mediumsized companies, researchers, and educational institutions in large numbers. In fact, no robot platforms are currently mass produced, which affects both pricing and availability. A research platform will only succeed if it’s beyond widely accepted; it must be affordable and available worldwide. The remainder of this article is a hands-on tutorial on the Franka Emika robot, the first series of industrial, AIready tactile robot platforms built not only for industry ●●

48

•

IEEE ROBOTICS & AUTOMATION MAGAZINE

•

JUNE 2022

but also robotics research and education with the aforementioned goals in mind. To see the platform’s scope on a technology level, the interested reader may refer to the official data sheet [1], which provides further details and references to the accordingly applied standards. Note that we refer to AI-readiness as the ability of the platform to 1) provide all sensory data (processed and raw) at the same 1-kHz real-time frequency used for internal control purposes. This includes measured joint data, such as joint/ Cartesian position; velocity as well as link-side jointtorque sensor signals; and an estimation of externally applied joint torques and wrenches as well as various further collision and contact information. Also, the forward kinematics of all robot links, Jacobian matrices of all robot joints, the inertia matrix, Coriolis, and centrifugal and gravity vectors are available to the maximum accuracy, 2) allow for low-level, desired joint-torque access at the same 1-kHz frequency used for internal purposes and with a very low latency, i.e., full customization with regard to physical manipulation, and 3) provide easy-to-access interfaces in common programming languages such as C++ and Python as well as connections to common research frameworks such as the Robotic Operating System (ROS) or MATLAB, which are widely used in robotics and AI research. The amount of proprioceptive low-level data that are available to feed AI-algorithms sums up to 2.3 MB/min. All of the tutorials shown in this work are also available in video form see [8]. The source code from the examples is available at [9]. Note that we use a single example pegin-hole task, namely, inserting a USB plug, throughout the article. For the readers who are already familiar with the basics of the platform and would like to, e.g., read about ML integration, they may skip ahead to the later sections. Related Platforms The suitability of a robotics research platform is first and foremost inherently tied to the technology powering it and the use cases it enables. The control paradigms, sensing technologies, low-level interfaces, and applications of the currently most widespread industrial manipulation platforms in research are summarized in Figure 2. On the joint level, robots that regulate only position or velocity, such as Universal Robot (UR) robots or ABB Yumi, neglect the dynamic characteristics of the joint. As a result, they operate with very stiff position control. To perform contact manipulation tasks such as insertion or polishing, they require an additional F/T sensor at the end effector, like e.g., the URe series. However, cost-effective F/T sensors typically provide low sensitivity and limit the range of applicable use cases, which remain mostly position/velocity based, such as pick and place or machinetending tasks. In contrast, joint-torque-controlled robots [10]–[12] regulate the joint-side torque, which explicitly incorporates the dynamic properties of the robot into the control loop. Joint-torque sensing is implemented either

•

+ Insertion, Polishing

Figure 2. A technological overview of the most relevant currently available robot manipulators, their control paradigms, sensors, interfaces, and target use cases. The robot images are taken from [2]–[7]. N/A: not applicable; UR: Universal Robot.

Sensitive Pick and Place, Sensitive Testing, Force Profile Tracking, Robust Fault Detection, Polishing, Grinding, Complex Sensitive Insertions, Screwing, Drilling, and Compliant Teaching Pick and Place, Machine Tending, Compliant Teaching, Compliant Insertions, Polishing, and Grinding Pick and Place, Machine Tending, Teaching With Pendant Use Cases

Position/Velocity, Joint Torque, Force, Collision Behavior Low-Level Interfaces

Position/Velocity

+ Force-Torque Sensor

JUNE 2022

+ Force

Current + Low Gear Ratio

Position/Velocity, Joint Torque, Force

Strain Gauges ML

Force Sensing

N/A

+ Admittance Control Coordination

N/A

Position/Velocity Control Control Joint Level

Current + High Gear Ratio

Series Elastic Actuators

N/A

Impedance Control

Joint-Torque Control

Strain Gauges

Learning Basic Assembly Skills Manipulation Signatures

Reflex and Impedance Control

Reflex and Joint-Torque Control

Franka Emika Robot Kinova Gen3 KUKA iiwa Barrett WAM Rethink Robotics Sawyer URe Series ABB Yumi UR Robots

by measuring the current combined with a low gear ratio [13] ( B ar re tt WA M A r m ) , by exploiting the implicit compliance of series elastic actuators [14] (Rethink Robotics’ Baxter), or by adding joint torque sensors [10]–[12] (KUKA iiwa). This enables controlling the robot’s impedance [15]–[20], i.e., its compliance as it interacts with the environment or with humans [12], [21], as well as the desired interaction force. In addition to basic impedance control, the Franka Emika robot implements all impedance, reflex, and tactile control features [22], including compliant teaching, safety [23], [24], collision and contact detection, and collision and contact reflexes [25]. It relies on highly sensitive straingauge-based torque sensors that, in addition to industrial-grade position performance, provide closed-loop force control performance capable of applying regulated forces as low as move_to_socket_pose set_move_configuration  } where parameter.socket_motion   –– set collisions thresholds   set_move_configuration insert  } where parameter.insertion_move_    configuration   –– Concurrent execution   barrier insert {   –> wiggle   –> press   –> monitor_pose  }   wiggle