Distributed autonomous robotic systems 8 [1 ed.] 9783642006432, 3642006434

Citation preview

Distributed Autonomous Robotic Systems 8

Hajime Asama, Haruhisa Kurokawa, Jun Ota, Kosuke Sekiyama (Eds.)

Distributed Autonomous Robotic Systems 8

ABC

Hajime Asama RACE (Research into Artifacts, Center for Engineering) The University of Tokyo Kashiwanoha 5-1-5 Kashiwa-shi, Chiba 277-8568 Japan E-mail: [email protected]

Jun Ota The University of Tokyo Graduate School of Engineering 7-3-1 Hongo, Bunkyo-Ku Tokyo 113-8656 Japan E-mail: [email protected]

Haruhisa Kurokawa Distributed System Design Research Group National Institute of Advanced Industrial Science and Technology (AIST) 1-2-1 Namiki, Tsukuba Ibaraki 305-8564 Japan E-mail: [email protected]

Kosuke Sekiyama Department of Micro-Nano Systems Engineering Nagoya University Nagoya 464-8603 Japan E-mail: [email protected]

ISBN 978-3-642-00643-2

e-ISBN 978-3-642-00644-9

DOI 10.1007/978-3-642-00644-9 Library of Congress Control Number: 2009922104 c 2009 Springer-Verlag Berlin Heidelberg  This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typeset and Cover Design: Scientific Publishing Services Pvt. Ltd., Chennai, India. Printed in acid-free paper 987654321 springer.com

Preface

The International Symposia on Distributed Autonomous Robotic Systems (DARS) started at Riken, Japan in 1992. Since then, the DARS symposia have been held every two years: in 1994 and 1996 in Japan (Riken, Wako), in 1998 in Germany (Karlsruhe), in 2000 in the USA (Knoxville, TN), in 2002 in Japan (Fukuoka), in 2004 in France (Toulouse), and in 2006 in the USA (Minneapolis, MN). The 9th DARS symposium, which was held during November 17–19 in Tsukuba, Japan, hosted 84 participants from 13 countries. The 48 papers presented there were selected through rigorous peer review with a 50% acceptance ratio. Along with three invited talks, they addressed the spreading research fields of DARS, which are classifiable along two streams: theoretical and standard studies of DARS, and interdisciplinary studies using DARS concepts. The former stream includes multi-robot cooperation (task assignment methodology among multiple robots, multi-robot localization, etc.), swarm intelligence, and modular robots. The latter includes distributed sensing, mobiligence, ambient intelligence, and multiagent systems interaction with human beings. This book not only offers readers the latest research results related to DARS from theoretical studies to application-oriented ones; it also describes the present trends of this field. With the diversity and depth revealed herein, we expect that DARS technologies will flourish soon. We thank everyone involved with the organization of DARS 2008. The members of the program committee organized sessions, reviewed papers, and contributed to enhancement of the quality of the program. We thank Prof. Alan Winfield (University of the West of England), Prof. Katsuhiro Nishinari (The University of Tokyo), and Prof. Gregory S. Chirikjian (The Johns Hopkins University) for their plenary and keynote talks. We truly appreciate co-sponsorship by the Mobiligence program of MEXT, and technical co-sponsorship by other organizations (IEEE RAS, SICE, RSJ, and JSME), and grants from the Inoue Foundation for Science and from the Suzuki Foundation. Kind assistance extended by the Tsukuba Convention Bureau was indispensable for us. We would like also to thank Dr. Kuniaki Kawabata, Dr. Masao Sugi, and Dr. Kohji Tomita for their invaluable help with local arrangements, conference website, and document edition. January 2009

Hajime Asama Haruhisa Kurokawa Jun Ota Kosuke Sekiyama

Contents

Part I: Distributed Sensing Multi-Robot Uniform Frequency Coverage of Significant Locations in the Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Marco Baglietto, Giorgio Cannata, Francesco Capezio, Antonio Sgorbissa Measuring the Accuracy of Distributed Algorithms on Multi-robot Systems with Dynamic Network Topologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . James McLurkin

3

15

Network Topology Reconfiguration Based on Risk Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kosuke Sekiyama, Hirohisa Araki

27

Energy-Efficient Distributed Target Tracking Using Wireless Relay Robots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chia Ching Ooi, Christian Schindelhauer

39

Cooperative Object Tracking with Mobile Robotic Sensor Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Junji Takahashi, Kosuke Sekiyama, Toshio Fukuda

51

Deployment and Management of Wireless Sensor Network Using Mobile Robots for Gathering Environmental Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tsuyoshi Suzuki, Ryuji Sugizaki, Kuniaki Kawabata, Yasushi Hada, Yoshito Tobe

63

VIII

Contents

Global Pose Estimation of Multiple Cameras with Particle Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ryuichi Ueda, Stefanos Nikolaidis, Akinobu Hayashi, Tamio Arai

73

Part II: Mobiligence Motion Control of Dense Robot Colony Using Thermodynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Antonio D’Angelo, Tetsuro Funato, Enrico Pagello

85

Modeling of Self-organized Competition Hierarchy with Body Weight Development in Larval Cricket, Gryllus bimaculatus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Shiro Yano, Yusuke Ikemoto, Hitoshi Aonuma, Takashi Nagao, Hajime Asama

97

Intelligent Mobility Playing the Role of Impulse Absorption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 Jae Heon Chung, Byung-Ju Yi, Chang Soo Han Part III: Ambient Intelligence Cognitive Ontology: A Concept Structure for Dynamic Event Interpretation and Description from Visual Scene . . . . . 123 Yuki Wakuda, Kosuke Sekiyama, Toshio Fukuda Subjective Timing Control in Synchronized Motion of Humans: A Basic Study for Human-Robot Interaction . . . . . . . 135 Mitsuharu Nojima, Hiroaki Shimo, Yoshihiro Miyake Robot Software Framework Using Object and Aspect Oriented Programming Paradigm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 Fumio Ozaki, Jun’ichiro Ooga, Kunikatsu Takase Distributed Context Assessment for Robots in Intelligent Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 Fulvio Mastrogiovanni, Antonio Sgorbissa, Renato Zaccaria Part IV: Swarm Intelligence Jamology: Physics of Self-driven Particles and Toward Solution of All Jams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175 Katsuhiro Nishinari

Contents

IX

Towards an Engineering Science of Robot Foraging . . . . . . . . . . 185 Alan F.T. Winfield A Modular Robot Driven by Protoplasmic Streaming . . . . . . . 193 Takuya Umedachi, Taichi Kitamura, Koichi Takeda, Toshiyuki Nakagaki, Ryo Kobayashi, Akio Ishiguro A Distributed Scalable Approach to Formation Control in Multi-robot Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203 I˜ naki Navarro, Jim Pugh, Alcherio Martinoli, Fernando Mat´ıa Guiding a Robot Flock via Informed Robots . . . . . . . . . . . . . . . . . 215 Hande C ¸ elikkanat, Ali Emre Turgut, Erol S ¸ ahin Theoretical and Empirical Study of Pedestrian Outflow through an Exit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227 Daichi Yanagisawa, Ayako Kimura, Ryosuke Nishi, Akiyasu Tomoeda, Katsuhiro Nishinari Understanding the Potential Impact of Multiple Robots in Odor Source Localization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 Thomas Lochmatter, Alcherio Martinoli Analyzing Multi-agent Activity Logs Using Process Mining Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251 Anne Rozinat, Stefan Zickler, Manuela Veloso, Wil M.P. van der Aalst, Colin McMillen Altruistic Relationships for Optimizing Task Fulfillment in Robot Communities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261 Christopher M. Clark, Ryan Morton, George A. Bekey Part V: Multi-robot Cooperation Robotic Self-replication, Self-diagnosis, and Self-repair: Probabilistic Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 Gregory S. Chirikjian Analysis of Multi-robot Play Effectiveness and of Distributed Incidental Play Recognition . . . . . . . . . . . . . . . . . . . . . 283 Colin McMillen, Manuela Veloso Sparsing of Information Matrix for Practical Application of SLAM for Autonomous Robots . . . . . . . . . . . . . . . . . . . . . . . . . . . 295 Haiwei Dong, Zhiwei Luo

X

Contents

Trajectory Generation for Multiple Robots of a Car Transportation System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305 Mitsuru Endo, Kenji Hirose, Yusuke Sugahara, Yasuhisa Hirata, Kazuhiro Kosuge, Takashi Kanbayashi, Mitsukazu Oomoto, Koki Suzuki, Kazunori Murakami, Kenichi Nakamura Distributed Control and Coordination of Cooperative Mobile Manipulator Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315 Enrico Simetti, Alessio Turetta, Giuseppe Casalino Rearrangement Task by Multiple Robots Using a Territorial Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325 Norisuke Fujii, Reiko Inoue, Jun Ota A Task Planner for an Autonomous Social Robot . . . . . . . . . . . . 335 Samir Alili, Rachid Alami, Vincent Montreuil A Stochastic Clustering Auction (SCA) for Centralized and Distributed Task Allocation in Multi-agent Teams . . . . . . . 345 Kai Zhang, Emmanuel G. Collins Jr., Dongqing Shi, Xiuwen Liu, Oscar Chuy Jr. Corridors for Robot Team Navigation . . . . . . . . . . . . . . . . . . . . . . . 355 Zack Butler, Carlos Bribiescas Part VI: Practical Control of Modular Robots Efficient Distributed Reinforcement Learning through Agreement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367 Paulina Varshavskaya, Leslie Pack Kaelbling, Daniela Rus Morphology Independent Learning in Modular Robots . . . . . . 379 David Johan Christensen, Mirko Bordignon, Ulrik Pagh Schultz, Danish Shaikh, Kasper Stoy Reconfigurable Modular Robots Adaptively Transforming a Mechanical Structure (Numerical Expression of Transformation Criteria of “CHOBIE II” and Motion Experiments) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393 Yosuke Suzuki, Norio Inou, Michihiko Koseki, Hitoshi Kimura Toward Flexible and Scalable Self-reconfiguration of M-TRAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405 Haruhisa Kurokawa, Kohji Tomita, Akiya Kamimura, Satoshi Murata

Contents

XI

Reconfigurable Teams: Cooperative Goal Seeking with Self-Reconfigurable Robots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417 Zack Butler, Eric Fabricant “Deformable Wheel”-A Self-recovering Modular Rolling Track . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 429 Harris Chi Ho Chiu, Michael Rubenstein, Wei-Min Shen Exploit Morphology to Simplify Docking of Self-reconfigurable Robots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441 Kasper Stoy, David Johan Christensen, David Brandt, Mirko Bordignon, Ulrik Pagh Schultz Reconfigurable Modular Universal Unit (MUU) for Mobile Robots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453 Shugen Ma, Changlong Ye, Bin Li, Yuechao Wang Part VII: Multi-robot Systems Design and Analysis for AGV Systems Using Competitive Co-evolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465 Ryosuke Chiba, Tamio Arai, Jun Ota Cooperative Control of Multi-robot Nonholonomic Systems with Dynamics Uncertainties and Control Time-Delays . . . . . . 477 Junjie Zhang, Suhada Jayasuriya Guaranteed-Performance Multi-robot Routing under Limited Communication Range . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 491 Alejandro R. Mosteo, Luis Montano, Michail G. Lagoudakis Pipeless Batch Plant with Operating Robots for a Multiproduct Production System . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503 Satoshi Hoshino, Hiroya Seki, Yuji Naka Control Methodology of Transfer Crane with Look-Ahead Rule in Harbor Transportation System . . . . . . . . . . . . . . . . . . . . . . 513 Hisato Hino, Satoshi Hoshino, Tomoharu Fujisawa, Shigehisa Maruyama, Jun Ota A Novel Marsupial Robot Society: Towards Long-Term Autonomy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523 Marek Matusiak, Janne Paanaj¨ arvi, Pekka Appelqvist, Mikko Elomaa, Mika Vainio, Tomi Ylikorpi, Aarne Halme

XII

Contents

Predicting the Movements of Robot Teams Using Generative Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 533 Simon Butler, Yiannis Demiris Interactive Mobile Robotic Drinking Glasses . . . . . . . . . . . . . . . . 543 Fran¸cois Rey, Michele Leidi, Francesco Mondada Part VIII: Human-Robot Interaction Adaptive Supervisory Control of a Communication Robot That Approaches Visitors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 555 Masahiro Shiomi, Takayuki Kanda, Kenta Nohara, Hiroshi Ishiguro, Norihiro Hagita Behavior Design of a Human-Interactive Robot through Parallel Tasks Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 565 Yuichi Kobayashi, Masaki Onishi, Shigeyuki Hosoe, Zhiwei Luo Tracking and Following People and Robots in Crowded Environment by a Mobile Robot with SOKUIKI Sensor . . . . . 575 Akira Ohshima, Shin’ichi Yuta Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 585

Part I

Distributed Sensing

Multi-Robot Uniform Frequency Coverage of Significant Locations in the Environment Marco Baglietto, Giorgio Cannata, Francesco Capezio, and Antonio Sgorbissa

Abstract. The article shows that the Random Walk and Edge Counting algorithms allow to solve – under some constraints – the Multi-Robot Uniform Frequency Coverage problem, i.e., the problem of repeatedly visiting a set of locations in the environment with uniform frequency. Both algorithms have extremely low requirements in terms of computational power and memory storage, and do not require inter-robot communication. Their properties are described in details and formally demonstrated.

1 Introduction This article considers a swarm of robots which are required to repeatedly visit a set of specific locations, a problem which is particularly relevant for surveillance and patrolling, continuous cleaning of crowded areas (e.g., malls, convention centers, restaurants), serving food or beverages (e.g., in hospitals or in a banquet), etc. As stated by a recent survey [20], the problem of Multi-Robot Uniform Frequency Coverage has received only a limited attention in literature. Traditional approaches rely on the idea of decomposing the space into subregions, and to consider each region separately in order to reduce complexity [21][1]. More recent approaches assume that robots are equipped with an ideal “sweeping tool”, and divide the area into a grid: the minimum spanning tree over the grid is built, and robots move along the Hamiltonian cycle that follows the tree around, therefore guaranteeing that each cell in the grid is visited repeatedly at the same optimal frequency [18][19]. The work described in this article1 has the additional constraints of finding solutions that are technologically plausible, with the purpose of designing robot swarms able to operate in the real-world here and now, not in a future when technology will Marco Baglietto, Giorgio Cannata, Francesco Capezio, and Antonio Sgorbissa DIST – University of Genova, Via Opera Pia 13, 16145, Genova, Italy e-mail: [email protected],[email protected], [email protected], [email protected] 1

The research leading to these results has received funding from the European Communitys Sixth Framework Programme under contract n 045255 - project ROBOSWARM.

4

M. Baglietto et al.

be – hopefully – available. Robots must be cheap, with low computational power and memory storage, and able to function even when wireless communication is not available or degraded, thus being unreliable for multi-robot coordination. Towards this end, one of the core technologies considered are passive RFID tags: these are low-cost, short-range, energetically self-sustained transponders that can be distributed in the environment and can store a limited amount of information. The use of RFIDs and other deployable sensor networks in robotics have been recently investigated, both to solve general robotic problems (i.e., navigation and localization [13][15][12]), as well as for specific applications (e.g., cleaning [10][11], exploration [8][9][2], or rescue applications in large scale disasters [16][14]). Both in the case that RFIDs are “manually” distributed during a setup phase which precedes execution, and in the case that robots themselves distribute RFIDs in the environment, they have been proven to provide a significant aid in different kinds of robotic tasks. In this work, it is assumed that RFIDs are distributed in the environment prior to robot operation, and used to build a “navigation graph”: each RFID stores navigation directions to neighboring RFIDs, thus allowing robots to safely execute paths from and to different locations. As a consequence of the technological constraints, we are interested in distributed, real-time search algorithms (either biologically inspired or not [7][6][3][4][5][2]), able to deal with situations in which a global representation of the environment, as well as global communication capabilities, are absent. Sections 2 shows that Random Walk and Edge Counting belong to a class of algorithms with common properties that allow, under some constraints, to solve the patrolling problem, and gives formal proofs of statistical completeness. Section 3 compares Edge Counting with Node Count (to our knowledge, the simpler algorithm for which formal proofs have been given [4]). Conclusions follow.

2 Multi-Robot Uniform Frequency Coverage 2.1 General Problem The MRUFC problem is defined as follows. Given that: • GN is a planar, non-oriented graph of arbitrary order, possibly containing cycles, which represents the topology of the free space, referred to as the Navigation Graph. As usual, the latter is better represented through an oriented graph Gˆ N , derived from GN by doubling all its edges and assigning them opposite directions. • S = {si } denotes the finite set of N vertices in Gˆ N . Each vertex si is associated with a location in the workspace. • Ai = {ai j } =  0 is the finite, nonempty set of (directed) edges that leave vertex si ∈ S. Each edge ai j is defined through a couple of indices (i, j), which respectively identify the corresponding start and end vertices. |Ai | is the dimension of the set, i.e., the number of edges departing from si . • R = {ri } is a set of M robots. Robots are allowed to move in the workspace from si to s j in Gˆ N only if ai j ∈ Ai , i.e., if the two vertices are adjacent.

Multi-Robot Uniform Frequency Coverage of Significant Locations

5

• Λ = [λ1 , · · · , λN ] is a vector which describes the average robot arrival rate to each vertex si ∈ S, expressed as number of robots divided by time (∀i, λi ∈ ℜ). The following objective must be achieved:

the M robots must guarantee continuous coverage of Gˆ N in such a way that ∀si , λi = λ¯ , where λ¯ is the same for all vertices.

This means that all vertices in Gˆ N are visited with the same frequency as time passes. For a real-world implementation, it is assumed that robots are equipped with proper algorithms for vertex-to-vertex navigation, as well as obstacle avoidance and localization. In particular, it is assumed that every vertex is linked to an adjacent vertex s j through ai j whenever it is possible to reach s j starting from si through a “simple motion”, e.g., moving on a straight line2 . The following additional implementation constraints are taken into account, which restrict the spectrum of possible solutions to the domain of the distributed and real-time search algorithms: • The algorithm which solves MRUFC must be executable in parallel on very simple robots with limited memory and computational power. This is achieved by assuming that robots do not know Gˆ N , neither in advance nor during navigation: Gˆ N is neither stored in robots memory, nor in a central repository. Instead, all the information concerning a generic vertex, as well as the edges departing from it, is stored into a smart node opportunely located in the environment 3 . To help robots to physically navigate in the workspace, every smart node stores navigation directions to reach neighboring smart nodes. • Robots and smart nodes have a very short communication range. Robots can directly communicate with smart nodes, and can indirectly communicate with other robots by writing to/reading from smart nodes. In no case, it is possible for a robot to directly communicate with another robot, for a smart node to directly communicate with another smart node, or for a robot to communicate with two smart nodes at the same time. In the following, starting from the experience acquired in previous work [17], two real-time search algorithms are described that are able to solve MRUFC (under restrictive conditions). Both algorithms, namely Random Walk and Edge Counting, assume that the M robots execute in parallel a particular instance of Algorithm 1, by properly implementing a strategy to choose the next vertex to be visited which varies depending on the particular algorithm considered. Algorithm 1 itself is straighforward. Line 1 chooses an arbitrary start vertex sstart , that can be different for different robots. When in vertex sc , the operator 2 3

The notion of “simple motion” varies depending on the robot kinematics, localization skills, etc. Smart nodes can be implemented, for example, as active or passive RFID tags, or similar devices with local communication capabilities and a very limited memory storage.

6

M. Baglietto et al.

Algorithm 1. Navigation Algorithm 1: sc = sstart 2: while TRUE do 3: ac j := choose(sc , Alg) 4: Move along edge ac j 5: sc := succ(sc , ac j ) 6: end while

choose(sc , Alg) in Line 3 returns one of the directed edges ac j ∈ Ac , according to a strategy that depends on algorithm Alg; in a real-world implementation, the operator choose(sc , Alg) requires the robot to communicate with the smart node which stores information about vertex sc , and to retrieve navigation directions that provide guidance to move towards the next vertex. Line 4 summarizes all procedures that are requested for the robot to move to the next node, including motion control, obstacle avoidance, localization, etc. As anticipated, it is assumed that the robot is able to move along edge ac j and to reach the next smart node correctly, i.e., it is equipped with proper hardware and software subsystems for achieving this. The operator succ(sc , ac j ) in Line 5 returns the successor vertex that results from the traversal of edge ac j ∈ Ac starting from vertex sc ∈ Gˆ N .

2.2 Random Walk and Edge Counting Both Random Walk and Edge Counting are able to solve a simpler version of MRUFC than that introduced in the previous subsection; in particular, they allow to achieve the following objective: the M robots must guarantee continuous coverage of Gˆ N in such a way that ∀si , λi = |Ai |λ¯ , where λ¯ is the same for all vertices. This means that all vertices si in Gˆ N which have the same number |Ai | of departing edges are visited with the same frequency as time passes. Remark that this property, demonstrated in the following, is not obvious: in fact, the frequency according to which each vertex is visited could depend, in theory, on the whole topology of the graph, thus making its computation difficult. Suppose that smart nodes are distributed in the environment to reflect the free topology of a typical office-like environment (Figure 1 on the left), the vertices of Gˆ N can be classified as: • terminal nodes, i.e., vertices with only one departing edge, corresponding to locations where a task must be performed (e.g., areas to be cleaned, locations where a snapshot must be taken, a restaurant table where food must be served, etc.). These correspond to vertices 1, 2, 5, 7, 8, and 9. • navigation nodes, i.e., vertices with more departing/arriving edges, corresponding to locations where smart nodes can provide a significant aid for navigation

Multi-Robot Uniform Frequency Coverage of Significant Locations

7

Fig. 1 Left: free-topology navigation graph GN . Right: implementation on smart nodes. The non-oriented graph GN is shown in place of Gˆ N for sake of clarity

and self-localization (e.g., T-Junctions, 90 degrees turns, doors, etc.). These correspond to vertices 3, 4, and 6. In this case, Random Walk and Edge Counting can be exploited to guarantee that, as time passes, all terminal nodes receive the same number of visits. Many realworld problems can be easily adapted to fit into this case. Figure 1 on the right shows the correspondence between vertices in GN and smart nodes (gray rectangles) in a free topology graph. Vertex 4 is a navigation node: if one wants that robots execute a task in its neighborhoods, it is necessary to add a new terminal node 10. Notice that, in this case, it is not necessary to add a new smart node, since the same smart node can store information about vertices 4 and 10, as well as edges which connect them to other vertices in GN and navigation directions. Algorithm 2. Operator choose(sc , Random Walk) 1: j = random(|Ac |) 2: return ac j ∈ Ac

Algorithm 3. Operator choose(sc , Edge Counting) 1: j = switchc 2: switchc = (switchc + 1) mod |Ac | 3: return ac j ∈ Ac

Given these general properties which are common to both algorithms, Random Walk and Edge Counting differ in how the operator choose(sc , Alg) is implemented: Random Walk selects one of the departing edges randomly with equal probability (Line 1 in Algorithm 2), whereas Edge Counting selects edge ac j depending on the current value of the variable switchc , which returns all admissible edges in round-robin in order to guarantee that robots leaving sc are equally distributed along all possible routes (Lines 1 and 2 in Algorithm 3). The computational requirements are minimal for both algorithms, since Random Walk requires a robot visiting sc to pick a random number, whereas Edge Counting requires the robot to update switchc . Memory requirements are less for Random Walk, since it requires to store in every smart node the set of departing edges Ai , whereas Edge Counting requires the additional variable switchi . In addition, every smart node must store navigation

8

M. Baglietto et al.

directions: however, the memory required for this does not depend on the actual implementation of choose(sc , Alg), and hence it is the same for all algorithms. Communication requirements are minimal for both algorithms: robots must be capable of local communication with the smart node which stores information about the vertex of the graph in which they currently are. When more robots are present, all robots execute Algorithm 1 in parallel. Notice however that, when using Random Walk, the choices performed by robots do not interfere with each other, but the same is not necessarily true in the case of Edge Counting. In fact, since switches are shared among all robots (i.e., every smart node stores a single switch), the decisions taken by a generic robot in sc depend on how many robots have visited the same vertex before. To deal with this problem, it is necessary to shift the focus of attention from robots to smart nodes: what matters are not the decision taken by single robots, but the average visiting rate to each vertex. In this perspective, the system can be modeled as a Closed Queueing Network, a dynamic model which allows to describe and analyze how service centers are allocated to customers in time, thus providing a powerful tool for system analysis in other application domains (e.g., process automation, communication networks, etc). Usually, Closed Queueing Networks can be modelled using Continuous Time Markov Chains (CTMC), with the purpose of describing the probability of being in a given state at a given time. However, since the MRUFC problem focuses on the average visiting rate, in the following it will be sufficient to consider flow balance equations for every vertex of the graph. This allows to refer to simpler mathematical tools than CTMC to describe the system’s behaviour at steady state. In particular, Closed Queueing Networks are specified by: • the number N of service centers si (corresponding to vertices of Gˆ N ); • the number M of customers (corresponding to robots); • the average arrival/departure rate λi to/from node si (i.e., expressed as number of arriving customers divided by time). The arrival and departure rate in si are the same in a Closed Queuing Network (and obviously correspond to the visiting rate), since customers cannot exit or enter the network from outside; • the number of servers mi running in parallel at si (i.e., how many robots can perform a task in si at the same time), and the average service rate µi at node si (expressed as number of robots which complete the task divided by time); • a routing policy, usually expressed through a N × N transition matrix P. If the transitions between vertices are probabilistic, as in the case of Random Walk, P is a stochastic matrix, and the generic element pi j ∈ P, with i, j = 1 . . . N, describes the probability that a robot in si heads towards s j after finishing its task. If the transition between vertices is deterministic, as in the case of Edge Counting, the generic element pi j ∈ P, with i, j = 1 . . . N, describes the percentage of robots that, after visiting si , head towards s j . About parameters mi and µi , notice that they are not considered in the following. Obviously, it is not possible for an infinite number of robots to be in the same node, and the time to perform the task associated with the node plays a fundamental role as well, especially in determining the traffic within an area. In theory, the fact that

Multi-Robot Uniform Frequency Coverage of Significant Locations

9

the node si is temporarily crowded should be modeled as if robots would be forced to wait in a queue before being served in si . Next, it would be necessary to check whether the stability conditions of the Jackson’s theorem, which guarantees that the length of the queue does not increase indefinitely, hold, that is, ∀i, λi < µi · mi

(1)

In the following, Equation 1 is assumed to be true, by choosing a number M of robots sufficiently small with respect to the dimensions of the workspace4. About the transition matrix P, notice that the following considerations are equally valid both when the generic element pi j ∈ P describes the probability of choosing ai j and when it describes the percentage of robots that chooses ai j : since P is used to express flow balance equations describing mutual relationships between average arrival/departure rates, the probability and the relative frequency according to which an edge is chosen assume the same meaning. In the following, the term transition probability is used also in the deterministic case for sake of brevity. By remembering that the probability pi j = |A1i | is identical for all edges departing from si , the transition matrix P turns out to be a non-symmetrical matrix with zeros in the diagonal since no vertex is connected to itself. ⎤ ⎡ 0 · · · p1 j · · · p1N ⎢ .. . . .. ⎥ ⎢ . . . ⎥ ⎥ ⎢ ⎢ (2) P = ⎢ p j1 · · · 0 · · · p jN ⎥ ⎥ ⎢ . .. ⎥ . . . ⎣ . . . ⎦ pN1 · · · pN j · · · 0

P is a stochastic matrix, since each row of P sums up to 1 (in general, this is not true for its columns). In addition, P is irreducible, since Gˆ N is strongly connected (i.e., it is possible to reach every vertex in a finite number of transitions). All these elements allow to introduce and demonstrate the following theorem.

Theorem 1: Given a Navigation Graph Gˆ N with N vertices; given M robots, each choosing the next vertex to be visited according to Random Walk or Edge Counting. The average rate of visits λi that each vertex si of Gˆ N receives as time passes is proportional to the number of incident edges |Ai | of si . • Proof: At steady state, the average arrival rate equals the average departure rate for every node. Then, it is possible to write flow balance equations: N

∀i = 1 . . . N,

λi =

∑ p ji λ j

(3)

j=1

Equation 3 can be re-written in matrix form as:

Λ = ΛP 4

It is assumed that the area is not as crowded with robots as to produce “traffic jams”.

(4)

10

M. Baglietto et al.

According to the Perron-Frobenius Theorem, the eigenvector problem above has a solution associated with the eigenvalue 1 as long as P is a stochastic irreducible matrix, which can be expressed as a function of one of the variables arbitrarily chosen, say λ1 = λ¯ . Since, for all vertex s j , p ji = 0 if and only if si is adjacent to s j , the system in Equation 3 can be rewritten as follows by referring to Equation 2: ∀i = 1 . . . N,

∑

λi =

j|a ji ∈A j

1 λj |A j |

(5)

It is shown now that the ∞1 solutions of the system in Equation 5 can be expressed as a function of λ¯ according to the following: ∀i = 1 . . . N,

λi =

|Ai | ¯ λ |A1 |

(6)

In fact, by substituting this expression of λi in Equation 5, it holds ∀i = 1 . . . N,

|Ai | ¯ λ= |A1 |

N

∑

j|a ji ∈A j

1 |A j | ¯ λ |A j | |A1 |

(7)

Remark that, for every i, the sum on the right side of the corresponding Equation has exactly Ai addends, i.e., corresponding to adjacent vertices. The fact that Equation 6 describes the ∞1 solutions to flow equations, is sufficient to say that the assumptions are valid: at steady state, ∀i the average rate of visits λi to si is proportional to the number of incident edges |Ai |. 

3 Experimental Results The system has been extensively tested, both in the MatLab environment and in the Player/Stage architecture. Since the focus of the paper is on the properties of Random Walk and Edge Counting algorithm, only MatLab simulations are shown here. 250

200

150

100

50

0

0

200

400

600

800

1000

Fig. 2 Left: a navigation graph considered in experiments. Right: number of visits vs. time with Random Walk (10 robots), start node = 1

Multi-Robot Uniform Frequency Coverage of Significant Locations 700

11

600

600

500

500 400 400 300 300 200 200 100

100

0

0

200

400

600

800

1000

0

0

200

400

600

800

1000

Fig. 3 Number of visits vs. time with 10 robots, sstart = 1. Left: Edge Counting. Right: Node Count

However, more realistic simulated experiments have been performed as well, with up to 20 Roomba robots equipped with an RFID antenna, as well as algorithms for RFID-based navigation, control and self-localization in presence of simulated sensor noise. Experiments shown here compare Edge Counting with Node Count, the simplest real-time search algorithm for which formal proofs of complete coverage (in a statistical sense) have been given. Node Count differs from Edge Counting in that it associates a value with each vertex of the graph, which counts how often each vertex has been already visited. When a robot enters a vertex, it increases the value of the vertex by one: it then moves to the adjacent vertex with the smallest value, that is, the adjacent vertex that has been visited the least number of times. Node Count and Edge Counting have been quantitatively compared by running them on many different, randomly generated graphs, by assuming that the time requested for vertex-to-vertex navigation is constant for all couples of vertices. As anticipated, this has no influence on the general properties of the algorithm, as long as we focus on the mutual relationships between average arrival rates to vertices. In the following, results corresponding to the graph in Figure 2 on the left are described in details. As expected, the results of experiments performed with Random Walk are similar to those obtained with Edge counting (an example is shown in Figure 2 on the right, in which each curves plots the number of visits received by a different terminal node in 1000 time steps). However, Edge Counting deserves a greater 30

40 35

25 30 20 25 15

20 15

10 10 5 5 0

0

20

40

60

80

100

120

0

0

20

40

60

80

100

120

Fig. 4 Transient behaviour, number of visits vs. time with 10 robots, sstart = 1. Left: Edge Counting. Right: Node Count

12

M. Baglietto et al.

25

45 40

20

35 30

15 25 20 10 15 10

5

5 0

0

20

40

60

80

100

120

0

0

20

40

60

80

100

120

Fig. 5 Transient behaviour, number of visits vs. time with 10 robots, sstart = random(N). Left: Edge Counting. Right: Node Count

attention since it has similar requirements in terms of RFID memory as Node Count, and proves to behave more efficiently than Random Walk in the short term. Figure 3 shows the number of visits to vertices versus time when 10 robots are present, all of them starting from node 1; plots correspond to Edge Counting (on the left) and Node Count (on the right). Each curve corresponds to a different vertex; since we are interested in the behaviour at steady state, the first 50 steps are not considered. It is easy to see that the peculiarity of Edge Counting is “regularity”, i.e., it visits terminal nodes with uniform frequency (corresponding to the set of “almost overlapping” curves with the lowest slope); the same property holds for vertices with 3 departing edges (the set of curves with intermediate slope), as well as for vertices with 4 departing edges (the set of curves with the highest slope). Nothing similar happens in Node Count; in fact, by zooming on Figure 3 on the right, it is easy to verify that it often happens that the same terminal node is visited 50 times before an “unlucky” terminal node is visited just 1 time. Figure 4 focuses on the transient behavior which corresponds to the former 120 steps, by considering only “terminal nodes”: since the 10 robots start from the same terminal node, it can initially happen that some vertices are visited more than others: however, the difference between the most visited and the less visited terminal node is always ≤ 15 when Edge Counting is used, and – most of all – this difference tends to be constant in time since the frequency in which terminal nodes are visited tends to be uniform. In Node Count the difference between the most and the less visited terminal node goes up to 30 within the first 120 steps, and – as already shown – it can even increase to higher values in the subsequent time steps. Figure 5 shows similar results, but assuming that each robot starts from a different vertex randomly chosen. In this case, the difference between the most visited and the less visited terminal node is always ≤ 5 when Edge Counting is used, whereas no significant increment in performance can be perceived in Node Count.

4 Conclusions The article has shown that the Random Walk and Edge Counting algorithms allow a swarm of robots to patrol an area without global representations of the environment

Multi-Robot Uniform Frequency Coverage of Significant Locations

13

or global communication capabilities. The algorithms are suited to scenarios in which some areas (i.e., terminal nodes) must be repeatedly visited to perform some task. A qualitative comparison with Node Count reveals that Random Walk and Edge Counting have desirable properties in terms of complete coverage and uniformity in terminal nodes visits, whereas Node Count has not such properties. One could object that Node Count, on average, performs a higher number of visits to each terminal node, even if the frequency is not uniform, whereas Edge Counting spends more time in navigation nodes (i.e., vertices with more than one departing edge). However, when in terminal nodes, it can be often assumed that robots perform a task which time is considerably higher than navigation time. If this happens, the time spent in vertex-tovertex navigation tends to become ignorable: Edge Counting still guarantees that the “duty time” is uniformly distributed among all terminal nodes, whereas Node Count does not. All algorithms work on very simple “memoryless” robots with limited computational power; neither algorithm require robots to explicitly communicate with each other. However, Node Count requires some information to be stored in the vertices: this is true for Edge Counting, but not for Random Walk. In neither case robots need to know the graph, since vertices as well as incident edges can be stored, for example, into smart nodes opportunely distributed in the environment. This allows to add new vertices to the graph in real-time (e.g., adding new smart nodes). Notice however that Node Count unrealistically requires that a robot in vertex s knows how many times neighboring vertices have been visited (which requires smart nodes to be very close to each other, or that they can communicate in some way), whereas neither Random Walk nor Edge Counting makes this very strong assumption. Finally, if the robot is able to recognize distinctive places in the environment (i.e., T-junctions, crossroads, turns), Random Walk can be used to guarantee uniform coverage basing exclusively on the environment’s topology, i.e., without needing smart nodes at all.

References 1. Choset, H.: Coverage for robotics - A survey of recent results. Annals Math. Artif. Intell. 31, 113–126 (2001) 2. Batalin, M.A., Sukhatme, G.S.: The analysis of an efficient algorithm for robot coverage and exploration based on sensor network deployment. In: Proc. IEEE Int. Conf. Robot. Autom. (2005) 3. Vaughan, R.T., Stoy, K., Sukhatme, G.S., Mataric, M.J.: Lost: Localization-space trails for robot teams. IEEE Trans. Robot. Autom. 18(5) (2002) 4. Koenig, S., Szymanski, B., Liu, Y.: Efficient and inefficient ant coverage methods. Annals Math. Artif. Intell. 41(76), 31 (2001) 5. Svennebring, J., Koenig, S.: Trail-laying robots for robust terrain coverage. In: Proc. IEEE Int. Conf. Robot. Autom (ICRA), pp. 75–82 (2003) 6. Wagner, I., Lindenbaum, M., Bruckstein, A.: Distributed covering by ant-robots using evaporating traces. IEEE Trans. Robot. Autom. 15(5), 918–933 (1999) 7. Balch, T., Arkin, R.: Avoiding the past: A simple, but effective strategy for reactive navigation. In: International Conference on Robotics and Automation, pp. 678–685

14

M. Baglietto et al.

8. Dudek, G., Jenkin, M., Milios, E., Wilkes, D.: Robotic exploration as graph construction. IEEE Trans. on Robot. and Autom. 7(6), 859–865 (1991) 9. Bender, M.A., Fernandez, A., Ron, D., Sahai, A., Vadhan, S.: The power of a pebble: Exploring and mapping directed graphs. In: Annual ACM Symposium on Theory of Computing (STOC 1998), Dallas, Texas, pp. 269–278 (1998) 10. http://www.vorwerk-teppich.de/sc/vorwerk/rfid_en.html 11. http://ebiquity.umbc.edu/blogger/2005/07/05/ robot-vacuum-guided-by-rfid/ 12. Tanaka, K., Kimuro, Y., Yamano, K., Hirayama, M., Kondo, E., Matsumoto, M.: A Supervised Learning Approach to Robot Localization Using a Short-Range RFID Sensor. IEICE Trans. on Information and Systems E90-D(11), 1762–1771 (2007) 13. Kim, M., Chonga, N.Y.: RFID-based mobile robot guidance to a stationary target. Mechatronics 17(4-5), 217–229 (2007) 14. Ziparo, V.A., Kleiner, A., Nebel, B., Nardi, D.: RFID-Based Exploration for Large Robot Team. In: Proc. of the IEEE Int. Conf. on Robot. and Autom., pp. 4606–4613 (2007) 15. H¨ahnel, D., Burgard, W., Fox, D., Fishkin, K., Philipose, M.: Mapping and Localization with RFID Technology. In: Proc. of the IEEE Int. Conf. on Robot. and Autom., pp. 1015–1020 (2004) 16. Kleiner, A., Prediger, J., Nebel, B.: RFID Technology-based Exploration and SLAM for Search And Rescue. In: Proc. of the IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, Beijing, pp. 4054–4059 (2006) 17. Baglietto, M., Cannata, G., Capezio, F., Sgorbissa, A.: PatrolGRAPH: a Distributed Algorithm for Multi-Robot Patrolling. In: Burgard, W., Dillmann, R., Plagemann, C., Vahrenkamp, N. (eds.) Intelligent Autonomous Systems 10 - IAS-10 (2008) 18. Gabriely, Y., Rimon, E.: Spanning-tree based coverage of continuous areas by a mobile robot. Annals of Mathematics and Artificial Intelligence 1(1) (2001) 19. Elmaliach, Y., Agmon, N., Kaminka, G.A.: Multi-Robot Area Patrol under Frequency Constraints. In: 2007 IEEE Int. Conf. on Robotics and Automation (April 2007) 20. Ota, J.: Multi-agent robot systems as distributed autonomous systems. Advanced Engineering Informatics 20(1), 59–70 (2006) 21. Ntafos, S.: Watchman routes under limited visibility. Comput. Geom. Theory Appl. 1(3), 149–170 (1992)

Measuring the Accuracy of Distributed Algorithms on Multi-robot Systems with Dynamic Network Topologies James McLurkin

Abstract. Distributed algorithms running on multi-robot systems rely on ad-hoc networks to relay messages throughout the group. The propagation speed of these messages is large, but not infinite, and problems in algorithm execution can arise when the robot speed is a large fraction of the message propagation speed. This defines a robot “speed limit”, as any robot moving away from a message source faster than the message speed will never receive new information, and cannot properly execute any algorithm. In this work, we focus on measuring the accuracy of multi-robot distributed algorithms. We define the Robot Speed Ratio (RSR) as the ratio of robot speed to message speed. We express the RSR in a form that is platform-independent and captures the relationship between communications usage, robot mobility, and algorithm accuracy, allowing for trade-offs between these quantities at design time. Finally, we present results from experiments with 30 robots that characterize the accuracy of preexisting distributed algorithms. In all cases, accuracy degrades as the RSR increases. In our experiments, a RSR of 0.005 allows good accuracy in all algorithms, a RSR of 0.02 allows reasonable accuracy in simple algorithms, and all algorithms tested are essentially useless at a RSR of 0.10 or higher.

1 Introduction and Related Work Distributed algorithms for multi-robot systems rely on ad-hoc networks to transmit messages further than an individual’s communication range. Algorithms require that messages travel through the network from source to destination, but the motion of the robots changes the network, which disrupts message propagation, ultimately effecting algorithm performance. In this paper, we measure the effects of robot mobility on algorithm performance. We focus on algorithms that use a message broadcast James McLurkin Massachusetts Institute of Technology, Cambridge, MA e-mail: [email protected]

16

J. McLurkin

flood to construct a spanning tree, but the general concepts we define should apply to any type of multi-hop communication. We define a dimensionless metric of robot speed appropriate for systems that rely on multi-hop communication. We empirically measure the accuracy of several distributed algorithms under different rates of network topology change. We find that the accuracy of most algorithms decreases as the robot speed metric increases. We consider three canonical algorithms in this paper: broadcast tree construction, message convergecast, and broadcast tree navigation. Broadcast tree construction[3] is a practical way to propagate a global message throughout the network. It is simple to implement, messages propagate rapidly, does not create cycles, and decays in an orderly fashion. However, if the tree construction process cannot maintain an accurate data structure in a dynamic network, algorithms that rely on the tree structure cannot operate properly. Convergecast is a common technique for accumulating global data onto a single network node [8, 9]. Each robot aggregates messages from its children in the tree, and passes this result towards the root. The aggregation step eliminates the need to route each message individually. This significantly reduces the total communications required, but requires an accurate routing tree to correctly propagate messages towards the root. Navigation algorithms can use the broadcast tree for physical routing [1, 7]. A requirement of these navigation algorithms is a strong correlation between the algorithmic tree structure and the physical tree structure.

2 Multi-robot Computational Model We focus on multi-robot applications that require highly mobile agents using local inter-robot communication. We define the state of an individual robot, a, as the tuple of its unique ID, its global pose, and its private and public variables. We define a configuration, C, as the collection of states of all n robots in the network. Each robot communicates its public variables with its neighbors within a fixed radius r. This produces a geometric graph, G, in which each robot is a vertex and the communications links between robots are edges. We assume that each robot has a sensor that can estimate the pose of its neighbors relative to its own coordinate frame. We do not assume each robot has access to its position in an external coordinate system. We do not address power utilization or conservation in this work, instead assuming that the robots have sufficient energy to allow unconstrained mobility and communication. We assume that all the robots regularly announce their public variables to their neighbors with a shared fixed period, τ , but with different individual offsets to prevent message interference. This defines a local round of computation; a period of time in which each robot receives an announcement message from each of its neighbors, processes these messages, and then transmits its own message. Globally, if we examine the state of the system at time t, then again at time t + τ , every robot will have completed one local round. This allows us to model group-level algorithm execution as proceeding in a series of discrete global rounds. We define algorithm accuracy as a metric that ranges between [0, 1] and quantifies the quality of a configuration with respect to a particular algorithm A.

Measuring Algorithm Accuracy in Dynamic Network Topologies

17

A configuration with an accuracy of 1 is an exemplar of the best possible performance of algorithm A, while an accuracy of 0 is the worst possible performance. The accuracy metric is tailored to each algorithm and application, and must be included in the algorithm specification.

3 The Robot Speed Ratio In the previous section, we assumed that the robots broadcast messages to their neighbors at regular intervals in order to maintain the network as they move. As the robot’s speed increases, so will the rate of network topology change, so the frequency of neighbor updates must also increase to maintain accurate connectivity information. Since communications bandwidth is limited, there is a maximum update rate, which correlates to some maximum robot speed. If the robots move faster than this speed, the network can no longer be properly maintained, and the performance of algorithms that rely on network communication will degrade. In this section, we define a metric that captures the relationship between robot speed and communications bandwidth. We want this metric to be dimensionless and able to be estimated a priori from basic system parameters, so that it can be used as a design tool for new systems and for algorithm performance comparisons across different hardware platforms. Our approach is to define the robot’s speed as a fraction of the maximum message propagation speed, smessage . Given the actual robot speed, srobot , we define the robot speed ratio, or RSR, as RSR =

srobot smessage

.

(1)

The robot speed is measured directly, but the propagation speed of the message through the network must consider buffering delays and channel capacity limits. Any robot moving away from a source of information, such as the root of a broadcast tree, with a RSR > 1 will not be able to receive new information from this source. This effectively disconnects the network, making it impossible for any distributed algorithm to run correctly. This is a conservative limit, as it only considers the speed of a robot. If the direction of motion is taken into account, motion towards or parallel to the motion of the source might not be subject to this limit. But this requires knowing the particular configuration and velocities of all the robots in the network, making it impossible to compute before the algorithm execution. We consider possible approaches to address this limitation in future work. To determine the robot speed ratio before execution, we need to estimate srobot and smessage . We will assume the actual speed of the robots does not deviate far from the commanded speed, and will use the latter for srobot in our calculations. We can d express the message speed as smessage = t hop , where dhop is the average distance hop a message travels away from the source per hop, and thop is the expected latency per hop. We compute dhop by noting that the configuration graph G is an r-disk

18

J. McLurkin

spanning subgraph of the fully connected graph, G f , which is formed by connecting all pairs of robots. The average spanning ratio, k, is the average stretch of the distance between any two pairs of robots a, b in graph G to the distance between the f same two robots in graph G f . If we let dab and dab be the distance robots a   between and b in graphs G and G f respectively, we can write k = mean

dab f dab

over all pairs

a and b. Using k, we express dhop as a fraction of the communications radius: dhop = kr . A network with a smaller spanning ratio supports paths that are closer in distance to the Euclidean distance. In the ideal case of no stretch, k = 1 and dhop = r, but this implies that a message can travel in a straight line, which is only possible with an infinite density of robots. We can estimate k a priori if we know the density of robots in the environment and the communication radius, r. For the configurations tested in this work, the average number of neighbors of a robot, mavg ≈ 10 and r ≈ 1, yielding k ≈ 1.4. See the reference by Kleinrock and Silvester [5] for a careful analytic treatment of the calculation of dhop as a function of node degree. The next step is to compute thop , the expected latency at each hop. We assume that while each robot uses the same round duration, τ , they all have different offsets, picked uniformly at random from 0 ≤ τoffset < τ . Therefore, the most delay between receipt and retransmit of a message by any robot is τ , and the minimum is σ , where σ is some small processing time. Rounding σ down to 0, we compute the expected time delay at each node as thop = τ2 . This gives us an expression for message speed d

of: smessage = t hop = k2rτ . hop However, the round duration τ is bounded from below by the minimum communication requirements of the algorithm under consideration, A. Given the maximum communication bandwidth of each robot, B, and an assumption on the maximum number of neighbors for each robot, mmax , and the communication requirements of a given algorithm A, we can compute the minimum round duration. We measure algorithm A’s communication requirements, BA , in bits per neighbor per round. We can then compute the smallest possible round, τmin = BA Bmmax . Note that this assumes that the physical communications medium is shared between neighboring robots and is half-duplex. This assumption is true for most of the radio and optical communications hardware used in a practical multi-robot system. Combining the above expressions for dhop, thop , and τ , we can express the message speed and the robot speed ratio as: smessage =

2rB srobot k mmax BA , RSR = k mmax BA 2rB

(2)

The RSR makes sense when we vary the parameters in the expression. Increasing r, the communication range, allows the message to travel further with each hop. Increasing B, the bandwidth, decreasing BA , the algorithm’s required bits per round, or reducing mmax , the number of neighbors, all allow the round, τ , to be shorter, which increases message speed. A smaller spanning ratio, k, gives the message a straighter, more efficient path. The maximum number of neighbors, mmax , can be

Measuring Algorithm Accuracy in Dynamic Network Topologies

19

estimated by knowing the workspace area and the number of robots, or controlled at run-time with dispersion algorithms. The designer has access to all of these quantities, and can estimate the RSR for an algorithm execution at design time, allowing for proper specification of components, algorithm designs, and other trade-offs.

4 Experimental Setup The SwarmBot [4] robot platform was used to validate algorithm performance. The robots are fully autonomous, using only local sensors and computation. Each robot has a 32-bit ARM processor running at 40mhz, a unique ID, and a bump sensor. Large top-mounted LEDs inform the user of the robot’s status. Each robot has an infra-red communication and localization system that allows nearby robots to communicate and determine the pose, p = {x, y, θ } of their neighbors relative to their local coordinate frame [10] . The system was run at its lowest power setting, which has a range of about one meter. The lowest power setting is used to produce multi-hop networks within the confines of our experimental workspace, which was an 2.43 m × 2.43 m (8’ × 8’) square. Ground truth was determined by a vision-based localization system developed by Newton Labs [6] that tracks the position, {x, y}, of each robot. The system uses an IR emitter on the top of each robot that encodes 10 bits of data per second per robot, allowing each robot to be uniquely identified and tracked. Mean positioning error was 15.4 mm, which was adequate to use as ground truth in our experiments. The system has an update rate of 1 hz, so we limited the maximum speed of the robots in all experiments to 80 mm/s.

(a) A SwarmBot

(b) The Swarm.

(c) Hop distribution on robots

Fig. 1 a. Each SwarmBot has an infra-red communication and localization system which enables neighboring robots to communicate and determine their pose, {x, y, θ } relative to each other. The three lights on top are the main user interface, and let a human determine the state of the robot from a distance. The radio is used only for data collection and software downloads. b. There are 112 total robots in the Swarm, but a typical experiment uses only 2535 at a time. c. Picture of the robots constructing a broadcast spanning tree in the experimental workspace. Robots that are an even number of hops from the root are flashing their blue light, those located an odd number of hops are flashing their red light. The root is in the lower-left

20

J. McLurkin

Each algorithm was run on 25-35 robots moving randomly around the environment. The motion behavior moves the robots in a straight line until they contact an obstacle, then uses the bump sensors to estimate the angle of incidence and “reflect” the robot back into the environment. The behavior works well at keeping robots dispersed throughout the environment, and the uncorrelated random motion changes the robot’s neighbors frequently, making it a good way to characterize how sensitive an algorithm’s performance is to changing configurations. We tested each algorithm in a static configuration and over a range of robot speed ratios from 0.005 to 0.640. This range of speeds is best plotted on a logarithmic axis, so we round the RSR of the static configurations up to 0.001 to plot on a logarithmic scale with the rest of the results. Each algorithm was tested at each speed long enough for multi-hop communications to complete and for performance to stabilize. This was no less that 10 diam(G) rounds for the two communications algorithms and from 5-10 successful completions of the navigation algorithm.

5 Communication and Navigation Algorithms Broadcast trees [3] (or communication gradients) underpin many other distributed algorithms, and understanding their performance in dynamic multi-robot networks is critical to building practical systems. In this section, we measure the performance of three algorithms: a broadcast tree root distance algorithm that estimates a robot’s distance from the root, a convergecast summation algorithm that aggregates global quantities onto a single robot, and a navigation algorithm that uses the broadcast tree for physical navigation structure. [1, 2, 11].

5.1 Broadcast Tree Root Distance The most basic use of a broadcast tree is to propagate messages from the root outwards. We can use these messages to estimate the distance from any robot to the root of the tree. Each robot uses its local pose measurements to find the distance to its parent in the tree, i.e. the distance the message has just traveled. The sum of these distances to any robot is the message path distance through the network, which is an estimate of the robot’s distance from the root, stretched by the average spanning ratio, k. Under ideal conditions, the topology of the network will closely match the actual positions of the robots, producing a distance estimate on each robot that is well-correlated to the actual distance of that robot from the root. Figure 1c is a picture of robots constructing a broadcast tree. The root robot is at the lower left-hand corner of the picture, and the parity of the hops from the root is illustrated with a blue or red light on the robot for an even and odd number of hops, respectively. This static configuration produced (fairly) concentric rings, as there is a strong correlation between the distance from the root and hops from the root. In our implementation, we add a public variable, treeDistance, to the state of each robot. The treeDistance of the root is 0. We examine a non-root robot a with parent

Measuring Algorithm Accuracy in Dynamic Network Topologies

21

Fig. 2 a: Root distance estimate vs. actual distance data. The distance estimate is well correlated with the Euclidean distance at low RSRs, but becomes increasingly uncorrelated as the RSR increases. The vertical lines in the plots were caused by a robot with faulty communications. b: The accuracy of the root distance estimate decreases as the RSR increases. c: Convergecast summation requires a stable network to route messages back to the root, and its accuracy also declines rapidly at high RSRs

22

J. McLurkin

→ We calculate a.treeDistance = p.treeDistance + − → p which has local pose − ap. ap. est The estimate of distance to the root on robot a is da = a.treeDistance. The path distance algorithm runs concurrently with the broadcast tree, and its results are calculated as soon as a robot receives a broadcast tree message. The total running time is O(diam(G)) rounds, and it uses O(m) computation per round, where m is the number of neighbors. The bits per round per robot required is O(m). The accuracy is the correlation coefficient computed between the actual and estimated root distances over all robots a ∈ C: ArootDistance = corrcoef(da , daest ), which returns a result in the range of [0, 1]. We compute the algorithm’s accuracy without compensating for the spanning ratio, as this global quantity can be difficult to measure locally. Therefore, we never expect to achieve an accuracy of 1 as the average spanning ratio always makes the estimated distance longer than the actual distance. In practice, this error is tolerable, usually less than 20%. Figure 2a shows plots of the root distance estimate vs. the Euclidean distance to the root for a variety of RSRs. Note that these quantities are correlated at low RSRs, and become increasingly uncorrelated at high RSRs. Figure 2b plots the accuracy of the path distance estimate over increasing RSRs. As the RSR increases, the correlation between the robot’s distance estimate and the actual distance decreases. The intuition is that robots moving at a high RSR can travel a large distance in each round of computation, carrying their previous hop counts and distance estimates with them a long distance. These errors affect neighbors in the next round, ultimately producing an inaccurate estimate.

5.2 Convergecast Summation A convergecast aggregates a global quantity, q, onto the root robot by using the tree as a routing structure to propagate partial sums to the root. Convergecast requires the tree to be stable while messages travel, and we would expect its performance to be sensitive to changing network topologies. In our implementation, we add a public variable, partialSum, to the state of each robot. We examine a particular robot a with k ≥ 0 children, c1 . . . ck . Each round, robot a computes its partial sum by adding the value of its quantity, qa , to the partial sum of each of its children: a.partialSum = qa + ∑ki=0 (ci .partialSum). The public variable, a.partialSum, is accessible to robot a’s parent. By this process, each robot in the tree computes the partial sum of q over the subtree rooted at itself. The subtree rooted at the root is the entire tree, so its partial sum is the total sum of q. In our experiment, we choose to compute the total number of robots in the configuration, so each robot sets their value of q = 1. The algorithm has a running time of O(diam(G)) rounds. The convergecast algorithm runs concurrently with the broadcast tree construction, and correct results reach the root in 2 depth(T) rounds after startup or any population change. There is O(m) computation per round, and the bits per robot per round required is O(m). The accuracy metric is defined in terms of the error in the value of q computed at −qactual | the root: Aconvergecast = 1 − |qmeasured , bound to [0, 1]. |q | actual

Measuring Algorithm Accuracy in Dynamic Network Topologies

23

broadcast tree navigation 1 0. 9 0. 8

accuracy

0. 7 0. 6 0. 5 0. 4 0. 3 0. 2 0. 1 0 0. 001

0. 010

0. 100

1. 000

RSR

(a) Navigation Paths

(b) Navigation accuracy vs. RSR

Fig. 3 a. Example paths from the broadcast tree navigation algorithm at a RSR of 0. The broadcast tree network is shown with grey edges. Note that it is not the minimal spanning tree, nor is it planar. Using this tree directly for navigation would produce poor results. b. Broadcast tree navigation accuracy vs. RSR. The algorithm did not function at a RSR of 0.08 or higher

The accuracy of the convergecast summation algorithm is shown in Figure 2c, and it degrades quickly as the RSR increases. This is expected, because the algorithm needs a stable broadcast tree to relay partial sums back towards the root robot. Because multiple cycles of broadcast and convergecast are happening simultaneously, the root may be receiving messages from other convergecasts that are “early” or “late”, depending on how the network was reconfigured by the robot’s motion. Our accuracy metric does not take this into account. A way to eliminate this error would be to tag each message with its initial round, and maintain several parallel convergecast summations over multiple rounds. However, we do not expect that this change would affect overall results appreciably,

5.3 Broadcast Tree Navigation The broadcast tree can be used to guide any robot in the configuration to the root by using the other robots in the network as navigational cues. We assume that the path between any two neighboring robots that can communicate is navigable. A robot can navigate to the root of the tree by always moving towards its current parent in the tree. In the process of moving towards its current parent, it will move to a new part of the network, and will select a new parent when it updates its tree structure. This repeats until the navigating robot reaches the root of the tree. See reference [10] for an empirical evaluation of this algorithm and reference [7] for a more theoretical treatment. In our implementation, the navigating robot a collects all j closer neighbors to the root into the set Pa . The navigating robot computes the midpoints between all the cyclically adjacent neighbors from Pa . Cyclically adjacent neighbors are two

24

J. McLurkin algorithm accuracy vs. RSR

algorithm accuracy vs. RSR 1. 00

1

0. 90

0. 9

0. 80

0. 8

0. 70

0. 50

accuracy

accuracy

0. 7 0. 60

0. 40

0. 6 0. 5

0. 30

0. 4

0. 20

0. 3

0. 10

0. 2

0. 00 0. 0010

0. 0100 convergecast summat ion

RSR

0. 1000

1. 0000

broadcast t ree navigat ion

broadcast t ree pat h dist ance

(a) Algorithms described in this paper.

0. 1 0 0. 001

0. 01

RSR

0. 1

1

(b) All algorithms tested using the RSR metric.

Fig. 4 a. The three algorithms described in this this paper. b. All algorithms tested using the RSR as an speed metric. The grey lines are from two algorithms that do not require their broadcast messages to have a tree structure or geometric correlation to physical positions, as a result, their performance is unaffected by changes in the RSR. The accuracy vs. RSR data for most of the other algorithms is fairly similar. This allows the metric to be used as a very coarse estimate of expected performance.

robots that are adjacent in the angular ordering of neighbors. The navigating robot selects the closest midpoint and moves towards it. Moving towards the midpoints between neighbors helps to reduce collisions. The motion of the navigating robot will cause its neighbors to change, and the set of neighbors in Pa is updated each round. If |Pa | = 1, then the robot has only one closer neighbor, its parent, and moves directly towards it. In this case, we rely on the low-level obstacle avoidance behavior to guide the robot around its parent, or any other obstacle. The key assumption is that the neighbors in Pa are actually closer to the root along the navigable path. Figure 2b shows that this correlation between the root distance estimate and the Euclidean distance degrades at high RSRs, so we would expect the accuracy of navigation to suffer as well. The accuracy of this navigation algorithm is best captured by the path efficiency: the ratio of the path actually traveled by the navigating robot to the shortest navigable path. For a given start and goal location in the external coordinate frame, the physical accuracy of the broadcast tree navigation s algorithm is: Anavigation ≡ soptimal . Paths closer to the optimal are more efficient and robot judged to be more accurate. Paths that are of infinite length, i.e. the navigating robot does not get to the goal position, have an accuracy of 0. Several example paths from experiments with RSR = 0 are shown in Figure 4a, and the accuracy of the algorithm is shown in Figure 4b. The algorithm performs well at low RSRs, but then fails completely at a RSR of 0.08 or above. This is consistent with our intuition that navigation accuracy relies on the broadcast root distance accuracy. When this requirement is not met, navigating robots cannot determine the correct direction to drive, and move in a random path, because the neighbors in Pa are not actually closer to the root.

Measuring Algorithm Accuracy in Dynamic Network Topologies

25

6 Conclusions and Future work The accuracy of all of the algorithms decreased as the robot speed ratio increased. In our experiments, a RSR of 0.005 allows good accuracy in all algorithms, a RSR of 0.02 allows reasonable accuracy in simple algorithms, and all algorithms tested are essentially useless at a RSR of 0.10 or higher. Quantitative comparisons between accuracies of different algorithms is limited, because each accuracy metric is user-designed and application-dependent. The accuracy metrics used in this work were all designed towards quantifying the relationship between the broadcast tree computational data structure and the broadcast tree network geometry. A different application using the same algorithms might have different accuracy metrics. Intuitively, the network stability required for each of these algorithms differs, with convergecast and navigation requiring a more stable network than the broadcast tree root distance. However, these differences are not visible in the data, instead most algorithms performance vs. RSR curves are similar. Further testing on different platforms is needed, but if this statement is true in general, then this would be a very useful tool to determine a coarse estimate of the accuracy of a particular algorithm on a particular system, even before the system is built or the algorithm is written. The space of performance can be roughly divided into three regions: one where the algorithm will almost certainly have high accuracy, one where the algorithm will almost certainly have low accuracy, and a transition region between the two where the technique cannot produce an accurate prediction. While the RSR is a convenient way to estimate the relationship between mobility and communications, it does not actually measure the changes in network topology which cause the accuracy degradation. There are many other metrics to capture topology changes, including “network half life”; the amount of time until half of the population has changed [12] or half of a nodes neighbors have changed, and “network churn”; for example, taking into account the the amount of time a node is in the network. Future work could develop a notion of “neighbor half-life” which could directly capture the rate of topology change for a given configuration control algorithm. However, it is not clear how to generalize this while avoiding the need to calculate different rates of change for different algorithms. Additionally, the RSR metric will only be proven useful after evaluation on different hardware platforms. The data confirms the intuition that communications bandwidth, robot mobility, and algorithm accuracy are interrelated. Given an accuracy vs. RSR plot for a particular algorithm, the designer can use the robot speed ratio in equation 2 to trade-off between these three quantities at design time, and evaluate performance at run-time. This can be a useful tool for the designer of multi-robot systems. Acknowledgements. We thank Leslie Kaelbling for many helpful discussions. This work is funded by grants from Boeing Corporation, and we are grateful for their generous support.

26

J. McLurkin

References 1. Batalin, M., Sukhatme, G.S., Hattig, M.: Mobile robot navigation using a sensor network, New Orleans, Louisiana, pp. 636–642 (April 2004) 2. Fang, Q., Gao, J., Guibas, L.J., de Silva, V., Zhang, L.: Glider: gradient landmark-based distributed routing for sensor networks. In: INFOCOM 2005. 24th Annual Joint Conference of the IEEE Computer and Communications Societies. Proceedings IEEE, vol. 1, pp. 339–350 (2005) 3. Intanagonwiwat, C., Govindan, R., Estrin, D.: Directed diffusion: A scalable and robust communication paradigm for sensor networks 4. iRobot. Swarmbot (2002), http://www.irobot.com 5. Kleinrock, L., Silvester, J.: Optimum transmission radii for packet radio networks or why six is a magic number. In: Proceedings of the IEEE National Telecommunications Conference, vol. 4, p. 14.3 (1978) 6. Newton Labs. Model 9000 vision system (2001), http://www.newtonlabs.com/9000.htm 7. Li, Q., Rus, D.: Navigation protocols in sensor networks. ACM Trans. Sen. Netw. 1, 3–35 (2005) 8. Madden, S., Hellerstein, J., Hong, W.: Tinydb: In-network query processing in tinyos. Intel Research, IRB-TR-02-014 (October 2002) 9. Madden, S.R., Franklin, M.J., Hellerstein, J.M., Hong, W.: Tinydb: an acquisitional query processing system for sensor networks. ACM Transactions on Database Systems (TODS) 30, 122–173 (2005) 10. McLurkin, J.: Stupid Robot Tricks: A Behavior-Based Distributed Algorithm Library for Programming Swarms of Robots. PhD thesis, Massachusetts Institute of Technology (2004) 11. Nagpal, R., Shrobe, H., Bachrach, J.: Organizing a global coordinate system from local information on an ad hoc sensor network. In: Proc. of Information Processing in Sensor Networks (IPSN) (2003) 12. Yan, T., He, T., Stankovic, J.A.: Differentiated surveillance for sensor networks. In: Proceedings of the 1st international conference on Embedded networked sensor systems, Los Angeles, California, USA, pp. 51–62. ACM, New York (2003)

Network Topology Reconfiguration Based on Risk Management Kosuke Sekiyama and Hirohisa Araki

1 Introduction A great deal of information and traffic circulate over the complex and largescale networks. Such examples abound in an airline route or road map, and the Internet web[1]. These are important social infrastructures and hence desired to possess sufficient resilience against an unpredictable breakdown in order to maintain the expected function. Including redundant nodes and links to the network is expected to allow more robust systems, however it would make the system more expensive and less efficient in the ordinary situations. Therefore, a well-balanced network topology taking into account both optimality and resilience is of great importance[2]. Recently, it has come to be known that a number of real world networks are scale-free network (SFN), as seen in the airline route map, electrical power network, and the Internet web[3, 4, 5]. One of the main features in SFN is that the network topology exhibits a power-law degree distribution: P (k) ∼ k −γ , where k is the number of link attached to a randomly chosen node in the network and γ is the scaling exponent. SFN is named for the fact that the power-law distributions do not have a median thad implies a typical size of the system[6]. It is also known that the average path length between the nodes are surprisingly small in SFN[7], hence the efficient transport is expected over the network. However, despite the fact that SFN is robust against random breakdown or removal of nodes, it is rather vulnerable to the intentional attacks to the hubs, in which links are concentrated[8, 9, 10, 11]. In this paper, we present an Kosuke Sekiyama Nagoya University Dept. of Micro-Nano Systems Engineering Furo-cho, Chikusa-ku, Nagoya 464-8603 Japan e-mail: [email protected] Hirohisa Araki University of Fukui, Bunkyo, Fukui, 910-8507 Japan e-mail: [email protected]

28

K. Sekiyama and H. Araki

autonomous reconfiguration model of network topology to maintain the balance between efficiency and robustness under the intentional attack to nodes. In a huge network, localized attacks to the network or breakdown are more likely to happen rather than the uniform and random breakdowns. In order to cope with the problem, an evolutionary network approach is applied in an attempt to strengthen the reliability and efficiency under the intentional attack. The intentional attack means that the hubs (i.e., centralized nodes) are more likely to be targeted by attacks according to the degree. Such an attack will lead disintegration of the network, hence more flat and redundant network topology is preferred, while a centralized network structure is more effective in a steady and secure condition. In this paper, modified preferential linking is introduced such that each node reconnect to the other nodes based on the evaluation of breakdown rate. Steady condition without node breakdown will lead to more centralized and cost-efficient network topology, while SFN or random network are obtainable in the case that intentional attack is present.

2 Optimal Network Topology by Rewiring Process 2.1 Model Outline Suppose a network composed of N nodes, which is depicted as a directed graph in fig.1(a). The solid line in the figure indicates an active connection from a sender to other nodes, while the dotted line indicates the acknowledge to the active connection. Where, a pair of the active connection and the acknowledge is regarded as one undirected link between the corresponding two nodes, supposing a situation in a communication network. The number of directed link that is allowed to be operated for each node is assumed to be L, then the number of corresponding acknowledge link from the linking node

(a) Start

(b) Breakdown

(c) Rewiring

(d) Recovery

Single Loop

Double Link

Fig. 1 Rewiring process due to node break- Fig. 2 Single loop and double down link

Network Topology Reconfiguration Based on Risk Management

29

is also L, hence the average link number for each node over the network is k = 2L. We exclude the single loop and double link as shown in fig.2. From what follows, the number of link for node is referred to as degree. Figs.1(a)-(d) show an outline of network reconfiguration process, where we presume a situation of breakdown and recovery in the comunication network. From the initial condition (fig.1(a)), a random breakdown or intentional attack occurs in a node, and the links to the breakdown node are disconnected (fig.1(b)). Then, the surviving nodes will attempt to rewire the connection from the breakdown nodes to another alive node. A node to be rewired is selected probabilistically from all available nodes in the network, avoiding the single loop and the double link structure (fig.1(c)). After a certain recovery time, the breakdown nodes are supposed to recover from a failure and included to the network, hence the network size is assumed to be constant over long time interval (fig.1(d)).

2.2 Rewiring Process of Network Topology The rewiring link process is defined to reconfigure the primary network topology, which is extension from the conventional preferential linking model [12, 13]. The preferential linking will select a node by the probability which is proportional to the degree of the node. This means that centralized nodes are more likely selected by the other nodes. Let ki be the degree of node i, then modified preferential linking from node i to j is given by the following equation, pij = 

(k¯j )β ¯ β. h∈Si (kh )

(1)

Where, Si is the node set available for rewiring from node i. For simplicity, Si is assumed to include all nodes except for the nodes which will conflict with the linking conditions of simple loop and double link in fig.2. Also, k¯j denotes time average of degree for the past T time step, t 1  kj (τ ). k¯j = T

(2)

τ =t−T

Reconfigured network topology is statistically characterized by degree distribution of the network. Some typical degree distributions are illustrated in fig.3. Also, β in eq.(1) indicates the extent of preferential strength. In the case of β = 1, eq.(1) reduces to the conventional preferential linking, and β = 0 means random selection, and a random network is obtainable as shown in fig.3(a). If β takes larger value, the reconfigured network topology approaches to a star-like network (fig.3(c)) through a scale-free network structure (fig.3(b)). Therefore, parameter β becomes a control parameter for a basic network topology configuration.

30

K. Sekiyama and H. Araki

100

100

100

10 1 1

10

100

1000

Link

(a) Random network

Degree

1000

Degree

1000

Degree

1000

10 1 1

10

100

1000

10 1 1

(b) Scale-free network

10

100

1000

Link

Link

(c) Star-like network

Fig. 3 Reconfiguration of Network Topology

2.3 Intentional Attack to Node According to the conventional research [14, 15, 16], robustness of the network is examined by deleting a node from the network one after another until the network is disintegrated. An intriguing result is that SFN is robust in that the network can remain integrated until nearly 50 % of nodes are removed from the network. However, it is unlikely that such a large part of the network is randomly deleted in reality. Therefore, we assume that a breakdown for node i occurs independently by the following probability, qi = min(vki + d, 1.0).

(3)

Where, d denotes the natural breakdown probability regardless of the degree k ∈ [L, N −1] and v denotes the coefficient that is related to intentional attack according to the node degree k. While v = 0 means uniform random attack, if v takes a larger value, the node with large degree is exposed to higher risk of breakdown. Such a breakdown pattern is referred to as an intentional attack. The hub nodes are more likely to be attacked with an increase of the probability qi . Thus, the optimal network topology has to be attained corresponding to the attack pattern.

3 Evaluation Index for Network Topology 3.1 Global Evaluation of Network Average Path Length In order to evaluate the efficiency of network performance, the average path length (APL) is generally employed. Although the accurate APL should be estimated based on the whole combination of the paths (N C2 ), we estimate APL by taking certain partial sample paths due to the computational cost. Let the optimal path length between the node pair (i, j) denote l(ij) , then the average path length E is defined as following,

Network Topology Reconfiguration Based on Risk Management

1 E= n

n 

lij

31

(4)

i,j,(i=j)

Increasing Rate of Average Path Length In general, there are several redundant paths to connect the distant nodes each other. In this model, we assume that when the path from node i to j is lost due to a breakdown, an alternative detour path ˆlij is established. Since ˆlij is considered less optimal than lij ; lij < ˆlij is expected. This process is an important feature of resilient network against an intentional attack, and the modified network has to be more robust against the same attack pattern. Now, in order to evaluate the robustness of the network topology, we introduce the APL increasing rate ∆E with the comparison of APL w.r.t. before and after node attack as follows, = 1 E n

n 

ˆlij

(5)

i,j,(i=j)

−E ∆E = E

(6)

If the node i and j are completely disconnected after the breakdown, we set ˆlij = N −1 for the descriptive purpose, where N is the maximum node number.

3.2 Local Evaluation of Network In addition to the global index, the local index of connectivity of the network is introduced. Let Ni the number of node which is reachable within 2-hop distance from node i, and suppose that some connections to the neighboring nodes are i after the breakdown. The lost due to a node attack. Then, Ni will change to N i as follows; local index is now defined as the normalized ratio of Ni → N Ri =

i N . Ni

(7)

Ri ∈ (0, 1) indicates the degree of local robustness to the breakdown.

4 Adaptive Network Toplogy Reconfiguration 4.1 Adaptive Rewiring Process Based on Evaluation Index Topology reconfiguration of the network is executed by rewiring links according to observation of breakdown pattern in the peripheral nodes. The modified preferential linking is employed, which is extension from the conventional

32

K. Sekiyama and H. Araki

preferential linking model [12, 13]. Firstly, the observed condition of node j is denoted by bj ∈ {0, 1} which takes bj = 1 in the case that a node breakdown occurs and otherwise bj = 0. The average of recent T step, ¯b is given by t  ¯bj = 1 bj (τ ). T

(8)

τ =t−T

Eq.8 denotes the observed time average breakdown rate for the node j and it is employed in the rewiring process. The rewiring probability is based on the form of eq.(1), however the coefficient β is extended so as to incorporate the breakdown situations. Therefore, such an extension is given by introducing a new control parameter as given by ηij = β exp[−γi¯bj ].

(9)

Where, β is now the constant which is specifically predefined so that the given preferential probability can regulate various network topologies: random, scale-free, and star-like structures. γi is a parameter to be adjusted as rewiring process proceeds. The new preferential linking rule is now given by (kj )ηij pij =  . (10) ηih h∈Si (kh ) The rewiring process, however, is driven by two different purposes. One is to repair the network topology for the node breakdown, and this process is referred to as passive rewiring. Another rewiring process is to optimize the network efficiency when serious breakdown is absent. Such a rewiring process is referred to as active rewiring. Passive rewiring The passive rewiring process aims to reconfigure the directed link which has been lost due to the breakdown of a connected node. Then the rewiring probability should be updated so that the network will evolve to more distributed and robust structure against the intentional attack. This is realized by increasing the value of γi in eq.(9). Let tn denote the discrete time step which is incremented when either of the active or passive rewiring process is executed, ¯ as follows; then γi is updated with R t 1  ¯ Ri (t) = Ri (τ ), T τ =t−T   ¯ e−Ri (tn−1 ) γi (tn−1 ). γi (tn ) = 1 + T

(11)

(12)

Network Topology Reconfiguration Based on Risk Management

33

The increasing rate of eq.(12) is attenuated by 1/T to fit the scale of variation between increasing and decreasing rate. Active rewiring When the network remains in safety and steady condition continues for a sufficient long period, the efficiency of network will be more emphasized than the robustness. The active rewiring process aims to reduce APL by rewiring the node connections aggressively. Active rewiring is executed in ever T step ¯ i = 1 and ∆Eij < Riskthreshold, only when R t 1  Ei (τ ). E¯i (t) = T τ =t−T   ¯ γi (tn ) = 1 − e−Ei (tn−1 ) γi (tn−1 ).

(13) (14)

The optimal γi is acquired based on the balance of both passive and active rewiring processes. Also, the meaning of Riskthreshold is discussed in the next section.

5 Network Reconfiguration with Risk Management Increasing diversity of the network will augment robustness against network attack while it may abate the network efficiency due to increasing average path length between the nodes. Taking into account of a breakdown risk, however, a sacrifice of the efficiency should be minimized. We have shown that the network topology can be reconfigured by changing the preferential linking probability (10) which is regulated by the value of γ in (9). By substituting several values to γ for respective attack pattern v, corresponding network is obtained. Figure 4 depicts a utility curve where the horizontal axis is the average path length E of converged network corresponding to the designated γ and the vertical axis is ∆E, which implies weakness; the average risk of increasing path length due to a single node attack. Table 1 shows γ values corresponding to the point number in fig.4. When small γ value is used, the more centralized and efficient network is configured but it is vulnerable to the node attack under the presence of heavy attack with large v. On the other hand, too large γ will sacrifice the network efficiency in the case of weak attack pattern with small v. So, γ should be decreased until the allowable risk Table 1 Plot point and corresponding γ value point γ

1 0.001

2 0.01

3 0.1

4 1.0

5 10.0

6 100.0

34

K. Sekiyama and H. Araki

Fig. 4 Utility curve in the presence of breakdown risk

600

12

Weakness

v = 0.001 v = 0.005 v = 0.01 v = 0.05

1

500

2

400

3

3

300 200 100

2

0 2

45 6 3

1

3 45 6 4

2.5

4 5

5 6

3 3.5 4 4.5 Optimal path length

6

5

Table 2 Intentional attack pattern v

d

time step

0.001 0.005 0.01 0.05

0.01 0.01 0.01 0.01

0 ∼ 2, 000 2, 001 ∼ 4, 000 4, 001 ∼ 6, 000 4, 001 ∼ 8, 000

Table 3 Simulation parameter N L T β RK

The number of node The number of directed link Time step for active rewiring Basic preferential linking parameter Allowable risk threshold

600 2 100 2 (100, 300)

threshold depending on attack patterns. Passive and active rewiring process will update γ so that it reaches an appropriate value for the attack pattern.

6 Simulations 6.1 Simulation Setup Simulations are conducted to confirm whether the most appropriate network topology is obtained based on the risk management with active and passive rewiring process. The attack pattern is changed in every 2000 step abruptly according to table2, and the simulation parameters are defined in table 3. Network reconfiguration process with active and passive rewiring procedure will function so that APL is decreased as long as the allowable risk threshold RK in table3 is satisfied. Also, the initial network topology is set to a random network.

Network Topology Reconfiguration Based on Risk Management 12

Weakness

2

400

200 100

2

0 2

45 6 Allowable risk

3

1

3 45 6 4

2.5

4 5

v = 0.001 v = 0.005 v = 0.01 v = 0.05

500

3

3

300

600

v = 0.001 v = 0.005 v = 0.01 v = 0.05

1

500

5

Weakness

600

3 3.5 4 4.5 Optimal path length

400 300

Allowable risk

200 100

6

6

35

0

5

2

(a)Risk = 100

2.5

3 3.5 4 4.5 Optimal path length

5

(b)Risk = 300

1000

1000

100

100

Degree

Degree

Fig. 5 Network Reconfiguration Process with Risk Management

10

1

10

1 1

10

100

1000

1

Step number

100

1000

Step number

(a)step = 2000

(b)step = 4000

1000

1000

100

100

Degree

Degree

10

10

1

10

1 1

10

100

1000

Step number

(c)step = 6000

1

10

100

1000

Step number

(d)step = 8000

Fig. 6 Degree distribution corresponding to Network Topology

6.2 Simulation Results Figure refoptimal(a) and (b) shows the trajectory of efficiency (APL: E) and weakness: average increased path length caused by the node breakdown (∆E). Fig. refoptimal(a) is the case where allowable risk threshold is set to 100, while Fig. refoptimal(b) is the case of RK= 300. For comparison, the default utility curve same with fig.4 is depicted. From both figures, it is confirmed that the reconfiguration process of network topology converged to the appropriate state where the APL is minimized under the constraint

36

K. Sekiyama and H. Araki

of allowable risk threshold. This implies centralization process and diversity for maintaining robustness is well organized under the presence of network attack pattern. Figures.6(a)-Fig.6(d) show the degree distribution corresponding to the reconfigured network topology for respective attack pattern. Figure 6(a) indicates the network is almost centralized form, while ig.6(d) implies that topology is almost random network because of hard intentional attack to the sink node. Thus, simulation results validated the network topology can consist with the demand of the risk management and the pursuit of efficiency coping with various attack patterns.

7 Conclusion We proposed a risk-management oriented network reconfiguration method based on the evolutionary network approach. Under the presence of intentional attack to the sink node, the balance of centralized and distributed network structure is modified so that it satisfies the predefined risk allowance.

References 1. Echenique, P., gomez Gardenes, J., Moreno, Y.: Dynamics of jamming transitions in complex networks. cond-mat/0412053 (2004) 2. Valente, A.X.C.N., Sarkar, A., Stone, H.A.: Two-peak and three-peak optimal complex networks. Physical Review Letters 92(2), 118702 (2004) 3. Huberman, B.A., Adamic, L.A.: Growth dynamics of the world wide web. Nature 401, 130 (1999) 4. Faloutsos, C.F.M., Faloutsos, P.: On power-law relationships of the internet topology. ACM SIGCOMM 1999, Review 29, 251–263 (1999) 5. Strogtz, S.H.: Exploring complex networks. Nature 410, 268–276 (2001) 6. Barabasi, A.L.: The New Science of Networks. Perseus Books Group (2002) 7. Cohen, R., Havlin, S.: Scale-free networks are ultrasmall. Physical Review Letters 90(5), 058701 (2003) 8. Albert, R., Jeong, H., Barabasi, A.-L.: Error and attack tolerance of complex networks. Nature 406, 378–382 (2000) 9. Cohen, R., Erez, K., ben Avraham, D., Havlin, S.: Resilience of the internet to random breakdowns. Physical Review Letters 85(21), 4626–4628 (2000) 10. Cohen, R., Erez, K., ben Avraham, D., Havlin, S.: Breakdown of the internet under intentional attack. Physical Review Letters 86(16), 3682–3685 (2001) 11. Zhao, L., Park, K., Lai, Y.-C.: Attack vulnerability of scale-free networks due to cascading breakdown. Physical Review E 70(035101) (2004) 12. GoLmez-Gardenes, J., Moreno, Y.: Local versus global knowledge in the barabalsi-albert scale-free network model. Physical Review E 69(037103) (2004) 13. Kalisky, T., Sreenivasan, S., Braunstein, L.A., Buldyrev, S.V., Havlin, S., Stanley, H.E.: Scale-free networks emerging from weighted random graphs. condmat/0503598 (2005)

Network Topology Reconfiguration Based on Risk Management

37

14. Motter1, A.E., Lai, Y.-C.: Cascade-based attacks on complex networks. Physical Review E 66(065102) (2002) 15. Crucitti, P., Latora, V., Marchiori, M., Rapisarda, A.: Error and attack tolerance of complex networks. Physica A 340, 388–394 (2004) 16. Guillaume, J.-L., Latapy, M., Magnien, C.: Comparison of failures and attacks on random and scale-free networks. In: Higashino, T. (ed.) OPODIS 2004. LNCS, vol. 3544, pp. 186–196. Springer, Heidelberg (2005)

Energy-Efficient Distributed Target Tracking Using Wireless Relay Robots Chia Ching Ooi and Christian Schindelhauer

Abstract. This paper considers the problem of mobile target tracking, in which a pursuer robot has to capture a mobile target with the aid of multiple wireless relay robots. We assume that the relay robots are able to self-organize to form an ad hoc robotic network when a tracking event is triggered, and act as static multihop relays to disseminate the latest target trajectory they detect to the pursuer. The task of the pursuer is to plan its path to capture the target based on the information received from the relays. Only the first position of target is given to the pursuer initially. No global information and no centralized infrastructure are provided. The heterogeneous robot team has to solve the problem using a distributed approach. We address this challenge by introducing Distributed Relay-robot Algorithm (DRA), which consists of two parts: information dissemination of target trajectory by relay robots using tAdv message and prediction of target position based on the first part. We study DRA using a network simulator and compare the results with CentralizedControl Approach (CCA). The outcomes show that DRA is more reliable, and it is feasible to use multihop communications to aid target tracking in a distributed way.

1 Introduction Recently, networked robotics has been gaining increasing attention among robotic and mobile networking research community. With the unprecedented growth of wireless communication technologies witnessed over the past two decades, many modern robots are already equipped with wireless communications capability [4, 2], using either radio frequency, for instance, bluetooth or IEEE 802.11-compliant [3] network interfaces, or infra-red communications. Multi-robot systems (MRS) [10] take the advantage of collective and cooperative behavior of robots through inter-robot communications. Swarm robotics [5] is an example of MRS that deploys and coordinates large number of relatively Chia Ching Ooi and Christian Schindelhauer Albert-Ludwigs-Universit¨at Freiburg, Computer Networks and Telematics, Georges-K¨ohler-Allee 51, 79110 Freiburg, Germany e-mail: {ooi,schindel}@informatik.uni-freiburg.de

40

C.C. Ooi and C. Schindelhauer

simple physical robots to complete assigned tasks as a team. These swarm robots can form a heterogeneous robotic team with larger, more powerful robots to cooperate to achieve common goals. Among them, mobile target tracking problem can be solved by employing such a heterogeneous robotic team. Due to their low power, the lightweight robots can act as static wireless relay robots to disseminate information of detected target, while the high power robots work as pursuers to chase and capture mobile targets. The co-location of the motion and task control systems with the communications mechanisms in a robot enables the co-exploitation of individual subsystems, providing a completely new kind of networked autonomy that is beneficial to many robotic applications. However, only little on-going researches make use of this advantage. Our paper explores this advantage to solve mobile target tracking problem using a team of heterogeneous robots communicating wirelessly. The remainder of the paper is organized as follows. Section 2 reviews related work on target tracking. In Section 3, we present the system and the energy models used in our research and define the problem of mobile target tracking using multiple wireless relay robots. Based on these foundations, we propose and analyze a distributed strategy called Distributed Relay-robot Algorithm (DRA) in Section 4. The proposed scheme is evaluated through simulation described in Section 5. Lastly, we make concluding remarks and discuss future works.

2 Related Work Target tracking problem has been a popular topic among the research community of Wireless Sensor and Actor Networks (WSANs) [6] since the past few years. Among them, binary-sensor network [7] introduces a binary sensor model, in which one-bit information is detected and broadcast to a base station. The one-bit information determines if an object is moving away or closer to the sensor, and therefore, target direction can be predicted. Using one-bit data is reasonable since sensor nodes have very limited energy and computational resources. However, it uses a centralized approach, with the base station keeping the bit information from every sensor. In [17], the concept of convoy tree based collaboration is introduced to track moving target. The target trajectory determines the addition and removal of sensor nodes from the convoy tree. This paper contributes mainly to the energy efficiency issue. Both of the papers do not consider the path planning of a pursuer robot. Pursuer-evader tracking is a specific tracking in which the evader always attempts to avoid pursuer by maximizing the distance between both nodes. The authors of [9] present a self-stabilizing approach that dynamically maintains a tracking tree of wireless sensor nodes rooted at evader. Only when the pursuer has found the tracking tree that it can follow the tree towards the evader. This paper restricts the speed of pursuer to be larger than the target speed. Moreover, the energy consumption for pursuer mobility is proportional to the distance traveled by target. In our proposed algorithm, we use target position prediction based on previous target trajectory information obtained from relay robots, leading to a more energy-efficient path of pursuer.

Energy-Efficient Distributed Target Tracking Using Wireless Relay Robots

41

On the other hand, [13] develops a hierarchical control system involving a multilayer data fusion in wireless sensor network. The control system has been demonstrated on a real sensor network deployment. However, a centralized controller is required to coordinate the pursuers to capture the evaders. Generally, a lightweight robot used in swarm robotics has higher power than a wireless sensor node. The wireless capability of robot enables it to act as a communications relay. To our best knowledge, no previous work addresses mobile target tracking problem using low power robots as wireless multihop relays, which assist a higher power pursuer robot in a distributed way. When a pursuer can obtain target information via the relay robots, a more efficient path of pursuer can be computed, leading to highly efficient energy and time consumption of the whole robotic team.

3 Preliminaries We consider an autonomous MRS that is employed to perform multiple tasks to achieve its assigned goals. The robot team is heterogeneous, and consists of both high and low power robots. Each robot is equipped with sensors to detect and identify moving objects and wireless data communications capability. Some tasks to be performed are event-driven. Among them, the robots have to form a mobile wireless network to track mobile target(s) in, for instance, an intruder tracking system. Due to the heterogeneity of the MRS and the different energy requirements for movement and radio communications, only the high power robots are assigned to act as pursuers, while the low power robots serve as wireless relays. In this scenario, we call the high power robot a pursuer and the low power robot a relay robot. The pursuer has to detect, chase and capture the moving targets with the aid of the relay robots. The energy required for robot movements [14] is generally much bigger than that for communications, if the communications do not involve high amount of data, e.g. multimedia data. According to [16], a wheeled vehicle with rubber tires at one kilogram moving on concrete must expend 0.1J/m to overcome 0.1N force of dynamic friction.On the other hand, depending on the transceiver sensitivity, the energy required by the power amplifier of transceiver to transmit one bit data over the distance of one meter ranges from some pico- to nano-Joule per bit per meterα 1 . We assume that every robot is aware of its own location by using localization [8] technique, and does time synchronization [15] prior to or during the tracking event. In addition, each relay robot is assumed to be able to detect and measure the target trajectory, including speed, direction, and the distance between the moving target and the relay robot using the motion detector, whenever the target is within a robot’s sensing range. In the next sub-sections, we introduce the system and the energy models, and define the problem of tracking a mobile target by a team of heterogeneous robots. 1

α is the path loss exponent of the transmission medium ranging from 2 to 6.

42

C.C. Ooi and C. Schindelhauer

Fig. 1 Mobile Target Tracking using Wireless Relay Robots

3.1 System Model The MRS consists of a single pursuer P and multiple relay robots. The tracking area of the MRS is modeled on a two dimensional Euclidean space as illustrated in Fig. 1. When a tracking event is triggered, the robots self-organize themselves to form a wireless ad hoc network [11]. Relay robots are static while the pursuer is mobile. Let Ns be the number of robots over the tracking region R ⊂ R2 , s0 ∈ R be the initial location of P, and si ∈ R be the position of i-th relay where i = 0, the robot set is defined as S = si : 0 ≤ i < Ns . Let Rc and Rs be the communication and sensing range of the robots respectively. G = (S, E) is a communication graph of the wireless robot  network such that (si , s j ) ∈ E if and only if ||si , s j ||2 ≤ Rc , where ||u, v||2 = (ux − vx )2 + (uy − vy )2 denotes the Euclidean distance between two robots. Let ts = 0 be the time the target first enters the sensing area of the MRS, and te denote the time the pursuer captures the target. We indicate the pursuer and target position as continuous function of time p : [0,te ] → R such that p p (t) and pq (t) denote the position of the pursuer robot and the mobile target respectively at time t in Cartesian Coordinates p(t) = (px (t), py (t)). When a target Q appears at pq ∈ R, robot i can detect Q if and only if ||si , pq ||2 ≤ Rs . Both the target and pursuer are moving at constant speed vq and v p respectively. When a relay robot detects a target, it transmits a message containing the target trajectory to the pursuer through multihop communications. Upon receiving the message, the pursuer predicts the target position based on the trajectory information retrieved from the message and the communication delay. Then, it computes a path towards the estimated target position. The proposed algorithm is described in Section 4.

3.2 Energy Model The robot team consumes energy for both communications and movement. We adopt the energy models used in [14]. The movement energy depends on the mass of the pursuer, the surface friction (air or ground), gravity and acceleration, and the distance traveled. We use the energy model for wheeled robot defined as Em = m · dm ,

Energy-Efficient Distributed Target Tracking Using Wireless Relay Robots

43

where the movement parameter, m, measured in Joule/meter, is a constant based on the aforementioned factors, and dm is the distance traversed by the pursuer in meter. On the other hand, the energy consumed to transmit ℓ bits of data over distance dc measured in meter is defined as Etx (ℓ, dc ) = ℓ · (dcα · etx + ecct ), where etx is the energy required by the power amplifier of transceiver to transmit one bit data over the distance of one meter, and ecct is the energy consumed in the electronic circuits of the transceiver to transmit or receive one bit, measured in the unit of Joule/bit. The energy consumption for receiving ℓ bits of data Erc (ℓ) = ℓ · ecct is independent of the distance between communicating nodes.

3.3 Mobile Target Tracking Problem Statement In this problem, a team of heterogeneous robots have to track and capture a mobile target with the aid of target information dissemination in a wireless ad hoc network. Definition. The mobile target tracking problem using multiple relay robots. Given a set of heterogeneous robots S that consists of one single pursuer robot P and Ns -1 relay robots. All robots have to form a wireless ad hoc network. The relay robots have to detect a mobile target Q moving at constant speed vq and disseminate the sensor information of target trajectory to the pursuer P. Using the information provided by relay robots, the pursuer P, moving at constant speed v p , has to compute its path to track and capture Q.

4 Algorithms Our tracking algorithm is composed of two parts: information dissemination of target trajectory and path planning based on future target position. The previous is performed by multiple relay robots while the latter is run by a pursuer robot P.

4.1 Information Dissemination of Target Trajectory Algorithm 1 is invoked and run in all wireless relay robots during the tracking event until the target Q is captured by the pursuer P. Each relay repeatedly senses if Q appears in its sensing range Rs . Let the relay robot detecting the target at time t be sq . When Q is detected, sq gathers the sensor information and produces three target trajectory information: target position, pq (t), target moving direction dirq (t), and target speed, vq (t). To disseminate these information to the pursuer P via multihop communications (through multiple relay robots), we introduce a target information advertisement message called tAdv. Other than the source address of sq and the destination address of P, a tAdv message consists of pq (t), dirq (t), vq (t) and t (the time Q is detected by sq ). Note that sq also detects if Q changes its trajectory, e.g. its direction. Upon any change detection, sq initiates a new tAdv to notify P about the latest target

44

C.C. Ooi and C. Schindelhauer

Algorithm 1. Target Info Dissemination: Wireless Relay Robot 1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18: 19: 20: 21: 22: 23: 24:

Input: motion sensor information Output: tAdv message initialize tAdvList repeat sensing Q if Q or its trajectory change is detected then generate and flood tAdv to P with initTtl update tAdvList end if if receive tAdv not in tAdvList then if P is my neighbor then forward tAdv to P update tAdvList else if ttl < maxTtl then ttl ⇐ ttl + incTtl flood tAdv with ttl to neighbors update tAdvList else discard tAdv end if end if end if until Q is captured by P

information. This is important to work with the prediction algorithm described in Section 4.2. For example, in case Q changes its direction before reaching the predicted position, P has to be informed to re-predict target position based on the latest tAdv contents. Moreover, instead of blind flooding, tAdv is disseminated based on the expanding ring search algorithm [11] that works as follows. The source node searches successively larger areas until the destination node is found, by expanding the search ring centered at the source node. In our algorithm, the relay robot increments the value of time-to-live (ttl) in every tAdv it receives for retransmission until P is found. Using IEEE 802.11 in static network, the ideal one-hop delay [12] for 11 Mbps data rate is around 2 ms, and about 7 times longer delay is expected for 1 Mbps. A tAdv list is maintained to avoid redundant rebroadcast of tAdv. A tAdv is generated by sq with ttl=initTtl. The relay robot receiving this tAdv increments the ttl by incTtl if P is not its neighbor and ttl has not reached maxTtl. Then, it forwards tAdv with the new ttl to all of its neighbors. This is repeated hopby-hop by the relay robots in the tracking region until P receives tAdv or maxTtl is reached. The value of maxTtl can be determined based on Ns upon the establishment of wireless ad hoc robot network [11].

Energy-Efficient Distributed Target Tracking Using Wireless Relay Robots

45

4.2 Prediction of Mobile Target Position P runs Algorithm 2 to compute its path to reach Q. At the beginning, P moves at v p towards the target position, pq (ts ), where it is first detected when it enters the tracking area at ts . We assume that P obtains the first target information by other means, for instance, through a centralized controller or base station, which also notifies the robot team of the tracking event. Whenever a tAdv is received, P invokes Algorithm 3 to estimate the future position of Q at pq′ and thus, plan its path to reach pq′ . This is repeated until P captures Q (or Q is within Rs of P). Algorithm 2. tAdv-based Path Planning: Pursuer Robot, P 1: 2: 3: 4: 5: 6: 7: 8: 9:

Input: First detected target position, pq (ts ) Output: Path plan for P pq′ ⇐ pq (ts ) repeat move to pq′ at v p if receive tAdv then run Algorithm 3 end if until Q is captured

Fig. 2 Prediction of Mobile Target Location

Algorithm 3 is explained next with the aid of Figure 2. P first checks the target detection time, tq,tadv in tAdv it receives. Only the tAdv with tq,tadv larger than that of previously processed tAdv will be handled. Otherwise, the tAdv is discarded. Two time-related parameters are important to predict pq′ : the multihop transmission delay of tAdv, tdelay and the traveling duration of P from its current position to pq′ , t pMove . tdelay is calculated based on tq,tadv and the time P receives tAdv, tcurr . On the other hand, P retrieves pq (t), dirq (t), vq (t) from tAdv to compute t pMove by using the law of cosines as follows. Let δ = p p (tcurr ), pq (tq,tadv ) 2 , β = (2·tdelay · δ · vq · cos θ ) − (vq ·tdelay )2 − δ 2 , α = v2q − v2p and γ = (2· v2q ·tdelay ) − (δ · vq · cos θ ), where θ is the angle between

46

C.C. Ooi and C. Schindelhauer

Algorithm 3. Prediction of Target Position, pq′ 1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18: 19: 20: 21:

Input: previous pq′ , latest Q detection time, tq,latest Output: predicted Q position, pq′ retrieve tq,tadv in tAdv if tq,tadv > tq,latest then tq,latest ⇐ tq,tadv tdelay ⇐ tcurr − tq,tadv get current position of P, p p (tcurr ) retrieve pq (tq,tadv ), vq , and dirq in tAdv compute p p (tcurr ), pq (tq,tadv ) 2 compute t pMove to find pq (tq,tadv + tqMove ) using law of cosines if t pMove > 0 then tqMove ⇐ t pMove + tdelay else tqMove ⇐ tdelay end if pq (tq,tadv + tqMove ) = pq (tq,tadv ) + vq ·tqMove pq′ = pq (tq,tadv + tqMove ) else Discard tAdv end if return pq′

p p (tcurr ) and pq (tq,tadv ) that is computed using arctan with these two positions.  When v p = vq , t pMove = β /γ . Otherwise, t pMove = −γ ± γ 2 + 4· α · β /(2· α ). Depending on the trajectory of Q, and the position of P and Q at ts , pq′ can not always be found based on t pMove . For instance, when vq is higher than v p and P is behind Q moving with linear mobility at tq,tadv . In this scenario, we base the computation of pq′ only on tdelay . Planning a path of P towards this pq′ enables P to receive more recent tAdv about Q’s trajectory from the relay robots. Once pq′ is computed, P continues with Algorithm 2 until Q is reached.

5 Performance Evaluation As described in Section 4, the proposed tracking algorithm computes a path for P based on tAdv contents disseminated in wireless networks. Thus, we have to consider communications parameters involved, such as transmission delay of tAdv. We evaluate the performance of the algorithm using the network simulator OMNet++ [1] throughout our work. We compare the results with another approach, in which P always knows the current position of Q without the aid of other robots. In the rest of the paper, we call this approach Centralized Control Approach (CCA) and our proposed algorithm Distributed Relay-robot Algorithm (DRA). Note that CCA assumes perfectly reliable and real time communications between P and its central controller. Two main criteria of our simulation study are the duration taken to

Energy-Efficient Distributed Target Tracking Using Wireless Relay Robots Parameter Value(s) Field size, (m2 ) 800 × 600 Number of pursuer 1 Number of relay robot 90 Number of target 1 Max. sensing range, Rs (m) 100 Max. transmission range, Rc (m) 100 Path loss exponent, α 3 Speed of P, v p (m/s) 2 Speed of Q, vq (m/s) 1, 2, 3, 4

47

Parameter Value(s) Energy consumed by transceiver circuitry 10−7 to transmit or receive a bit, ecct (Joule) Energy consumed by transceiver amplifier 10−12 to transmit one bit data over one meter, etx (Joule) Energy to receive a bit, erc (Joule) 10−7 ttl: Initial, Incremental, Maximum 1, 1, 10 tAdv Data Size (bits) 48 Physical, MAC, Network Layer Header Size (bits) 192,272,160 Energy to move P over one meter, m (Joule) 3

Fig. 3 Simulation Parameters

capture Q and the total energy consumed by all robots (considering both mobility and communications costs).

5.1 Simulation Setup We simulate 90 static relay robots cooperating with a pursuer robot on an area of 800m×600m. Only one mobile target is simulated. All robots apply IEEE 802.11 as their MAC layer protocols and no special routing protocol is used. The flooding area of tAdv is controlled by expanding ring search as explained in Section 4.1. Figure 3 shows the simulation parameters. For simplification, we fix Rs and Rc at 100m, which is applicable to both IEEE 802.11 and some existing motion detectors. As mentioned in Section 3, m at 3 J/m is suitable for wheeled robots at 30-kg moving on flat surface. Depending on the application, a lower surface friction still allows a lighter weight robot to act as a pursuer. At the beginning, all relay robots are distributed uniformly and randomly over the simulation area. Once deployed, a robot connects itself to its neighbors within its communication range. The mobility of Q is simulated using Rectangle Mobility to show the impact of its direction change to tAdv generation by relay robots(s). A simulation run is terminated whenever Q is detected within Rs of P.

5.2 Simulation Results Using the simulation parameters in Figure 3, we perform 50 simulation runs for each vq and compute total energy consumed by the robot team, and total time P takes to capture Q using both DRA and CCA, and compare the two results. First, we show two example results of a simulation scenario in Figure 4, using DRA and CCA strategies in Figure 4(a) and 4(b) respectively. The initial robot-target distance is set at 406 m. Using DRA, total time and energy consumed are 291.40 s and 1782.54 J, in which communication cost constitutes only 1.27 J. Due to space limit, we will not display the resulting communication cost expended. In all simulations, wireless communication cost is negligible compared to movement cost.On the other hand, CCA consumes 286.28 s and 1717.802 J of movement energy. These results show that the proposed distributed DRA strategy needs only an additional 1.76%

48

C.C. Ooi and C. Schindelhauer

(a)

(b)

400

Average Total Movement Energy of Pursuer (Joule)

Average Total Time consumed by Pursuer to detect Target (seconds)

Fig. 4 Example Results using (a) DRA and (b) CCA: Pursuer (on blue track) and Target (on rectangular black track) DRA CCA

350

300

250

200

2400 2200

DRA CCA

2000 1800 1600 1400 1200 1000 800

150 0.5

1

1.5

0.5

2

(a)

1.5

2

(b)

1100

2

DRA 1000 CCA

DRA CCA

900 1.5 800

% of Tracking Failure

Average Total Traveling Distance of Target (meter)

1

Ratio of Target Speed to Pursuer Speed

Ratio of Target Speed to Pursuer Speed

700 600 500 400

1

0.5 300 200 100

0 0.5

1

1.5

Ratio of Target Speed to Pursuer Speed

(c)

2

0.5

1

1.5

2

Ratio of Target Speed to Pursuer Speed

(d)

Fig. 5 Comparison of Tracking Strategies using Distributed DRA and Centralized CCA: (a) Average Total Time Consumption (b) Average Total Movement Cost of P (c) Average Total Traveling Distance of Q (d) % of Tracking Failure (pursuer failed to reach target within 1500s)

time and 3.63% energy compared to the centralized approach.Moreover, if CCA involves long-range communications (ignored in our simulation) with the centralized controller, e.g. a base station, additional communication cost and delay have to be considered. Figure 5 illustrates the comparison results of the proposed distributed approach, DRA and the centralized CCA strategy. To simulate various types of target being tracked, we vary the target speed from 1 to 4 m/s, while pursuer speed is fixed at 2m/s. First, in Figure 5(a), we compute and compare the average amount of time consumed by each algorithm to capture the target, with both approaches using the same scenario setting. Initial position of P is the same for all simulations, and Q

Energy-Efficient Distributed Target Tracking Using Wireless Relay Robots

49

always starts moving away from the first position of P. Thus, it is reasonable to observe that the average tracking duration reduces over higher speed ratio since the target mobility is based on rectangle mobility that Q goes back to its origin towards completing one round. Overall, CCA performs 4.49% to 22.7% better than DRA considering that DRA does not provide instantaneous target information due to transmission delay of tAdv, while in CCA, P always knows the current position of Q, with an exception at ratio 1.5. We deduce that the performance gap between DRA and CCA decreases for scenarios with speed ratio ≥ 1. This is because the prediction algorithm of DRA described in Section 4.2 takes target speed as one of the factors for computing future target position. Figure 5(b) shows the average mobility cost of P to capture Q. Since the robot speed is constant and fixed over all simulation runs, based on the energy model described in Section 3.2, the result is identical to that in Figure 5(a). Figure 5(c) shows the average total distance traveled by Q. This metric is important for tracking application that aims at minimizing the total distance traveled by a target, for instance, due to the larger loss an intruder may cause the further it manages to travel in the tracking area. The increasing total distance over higher target speed can be observed in both approaches, which show a maximum difference of only 48.64 meter, except in the case v p = vq . An appropriate approach can be chosen based on the application requirement. As some tracking applications require the pursuer to capture the target within an allocated time, we set the maximum simulation duration at 1500 s. Figure 5(d) shows that DRA performs without any failure, while CCA has the failure rates of 2% and 1% at vq =3 and 4 respectively. Note that all failed cases have been excluded from the previous statistics to fairly compare both approaches. CCA approach always directs the pursuer to current target position without considering the target trajectory. On the other hand, DRA predicts target position in advance based on the last detected target speed.

6 Conclusions and Future Works In this paper, we study the feasibility of using wireless communications of a robotic team in a mobile target tracking application. We introduce a distributed approach, DRA by employing low power robots as wireless relays. By predicting target position based on the target trajectory information supplied by relay robots, the pursuer robot computes its path towards capturing the target. The results of the simulations performed using a network simulator show that it is feasible to use multihop communications to assist target tracking in a distributed means. Lastly, DRA is more energy-efficient because observing the much lower communication cost compared to mobility cost, it is beneficial to take the advantages of relay robots to provide useful target information for pursuer to compute an energy-efficient path. As the communication cost is much lower than the movement cost, it is promising to develop an application-specific ad hoc routing protocol to maintain a multihop route between pursuer and relays closest to the target, which distance should

50

C.C. Ooi and C. Schindelhauer

decrease over time. We have also found several variations of mobile target tracking problem worthy of study using robots as relays. They include changing the number of pursuer and target, utilization of relay mobility, and various target trajectory impacted by using different mobility models, and lastly, the case in which a target acts as an evader. Acknowledgements. This research is partly supported by DFG-Sonderforschungsbereich SPP 1183: Organic Computing. Smart Teams: Local, Distributed Strategies for SelfOrganizing Robotic Exploration Teams.

References 1. OMNet++, discrete event simulation environment, http://www.omnetpp.org 2. EPFL Education Robot (2007), http://www.e-puck.org 3. IEEE 802.11 Wireless Local Area Networks - The Working Group for WLAN Standards (2007), http://www.ieee802.org/11/ 4. K-Team Corporation (2007), http://www.k-team.com 5. Swarm-robotics.org (2007), http://www.swarm-robotics.org/ 6. Akyildiz, I.F., Kasimoglu, I.H.: Wireless sensor and actor networks: Research challenges. Ad Hoc Networks (2004) 7. Aslam, J., Butler, Z., Constantin, F., Crespi, V., Cybenko, G., Rus, D.: Tracking a moving object with a binary sensor network. In: Proceedings of ACM SenSys, pp. 150–161 (2003) 8. Bachrach, J., Taylor, C.: Localization in sensor networks (2005) 9. Demirbas, M., Arora, A., Gouda, M.: Pursuer-evader tracking in sensor networks. Sensor Network Operations 5, 587–608 (2007) 10. Farinelli, A., Iocchi, L., Nardi, D.: Multirobot systems: A classification focused on coordination. IEEE Trans. Syst., Man, Cybern (2004) 11. Ilyas, M., Dorf, R.C. (eds.): The handbook of ad hoc wireless networks. CRC Press, Inc., Boca Raton (2003) 12. Li, F.Y., Hafslund, A., Hauge, M., Engelstad, P., Kure, O., Spilling, P.: Dilemma of using high datarate in ieee 802.11b based multihop ad hoc networks. In: International Conference on CIIT (2004) 13. Oh, S., Schenato, L., Chen, P., Sastry, S.: Tracking and coordination of multiple agents using sensor networks: System design, algorithms and experiments. Proceedings of the IEEE 95(1), 234–254 (2007) 14. Ooi, C.C., Schindelhauer, C.: Minimal energy path planning for wireless robots. In: ROBOCOMM (2007) 15. R¨omer, K., Blum, P., Meier, L.: Time synchronization and calibration in wireless sensor networks. In: Stojmenovic, I. (ed.) Handbook of Sensor Networks, pp. 199–237. John Wiley and Sons, Chichester (2005) 16. Tipler, P.A.: Physics For Scientists and Engineers. Worth Publishers (1991) 17. Zhang, W., Cao, G.: Dctc: dynamic convoy tree-based collaboration for target tracking in sensor networks. IEEE Transactions on Wireless Communications 3(5), 1689–1701 (2004)

Cooperative Object Tracking with Mobile Robotic Sensor Network Junji Takahashi, Kosuke Sekiyama, and Toshio Fukuda

Abstract. This paper deals with a cooperative object tracking with mobile robotic sensor network. The occlusion problem, which is one of the most important problems of mobile object tracking, will be solve with utilizing the mobile robotic sensor network. Additionally, the high quality information of the object can be observed, since the robotic sensor network can change observation formation their own way. On the other hand, to solve these problems the high cognitive capability and high intelligence sharing ability of robots are required. Consequently, we propose to use high intelligent group robot each of which can communicate with Peer-to-Peer communication. The self-deployment algorithm and network routing algorithm are developed as a base technic of mobile sensor network. The validity of these proposed algorithms are shown by multihop teleoperation experiment. The architecture to share common perceptions which is necessary for cooperative work are proposed.

1 Introduction The supervising system is very effective for preventing of crime, and vision sensors have been installed to various public space. The target tracking technic will be realized by using multiple cameras network[1, 2, 3]. On the other hand there is occlusion problem. When a fixed camera is under occlusion, in principle, it can not find out itself under occlusion. In other words, when a camera does not see the target, a camera can not judge which reason is right: (1)because the target object does not exist, (2)because there is a obstacle between itself and the target. However, if there are other robot, one of which looking at the occlusioned camera and obstacle can tell the camera that it is under the occlusion. So, for avoidance the occlusion problem, multiple robot system equipped with camera are required. Junji Takahashi, Kosuke Sekiyama, and Toshio Fukuda Department of Micro-Nano Systems Engineering, Nagoya University, 1 Furo-cho, Chikusa-ku, Nagoya, Aichi, Japan e-mail: [email protected], [email protected], [email protected]

52

J. Takahashi et al.

Various distributed algorithm have been proposed for multi-robot coodination for target tracking. The ALLIANCE architecture[4] achieves target-assignment by the interaction of motivational behavior. Black and Ellis[5] propose using the ground plane holography method to get 3D position of the target. Jung and Sukhatme[6] proposed an algolithm based on treating the densities of robots and targets as properties of the environment. The purpose of this research is to develop the object tracking system with mobile robotic sensor network. In order to realize this purppose, elemental technologis as follows are aquired. First is mobile robotic sensor network system in which robot can communicate with peer-to-peer. Second is self-deployment algorithm for robots deploying to suited formation. Some self-deployment algorithms using potntial field have been proposed[7]. Also fluid dynamics approach has been proposed for scalable self-deployment of mobile sensor networks[8]. However, these algorithm can not make a network various formation. Third is network routing algorithm for robots communication. Fourth is recognition algorithm using vision sensor. Fifth is sharing common perception method between robots. Sixth is cooperative decision making algorithm for task allocation. In this paper, robot system, the self-deployment algorithm and network routing algorithm are presented mainly.

2 Intelligent Group Robot System The experimental mobile robot is shown in Fig. 1(1),(2). The mobile robot basically consists of three modules: the decision making module is a PC (VAIO type-U, Sony Coro.), the Peer-to-Peer communication module is a ZigBee device, and the mobile module is an Pioneer3 or Amigobot (MobileRobots Inc.). Especially, the robots observing and tracking object target are equipped with camera. The robot control system is developed based on thread technique. So, each devices can be controlled simultaneously by one executive program.

2.1 Robot Control System 2.1.1

Robot System

The control program of the robot system is an executable file running on windows. This program consists of a Sensing-threadCa Communication-threadCa MovementthreadCand a Console-thread, each of which runs and accesses the main memory asynchronously. Communication between the threads is executed through the main memory. For example, the communication-thread writes the information of other robots sending packets on the main memory. At the same time, the movement-thread reads the information written in the main memory in order to decide the connection priority or movement direction.

Cooperative Object Tracking with Mobile Robotic Sensor Network

(1)

53

(2)

Fig. 1 Robot system and group robot system

2.2 Sink Controller for Monitoring the Mobile Sensor Network The sink controller is shown in Fig. 1(2) (the detail is in Fig. 7). It shows the position of all robots and the communication links. An operator can monitor the network easily by this window. Furthermore, target information such as position and propaties are should be drawn on this window. In addition, an operator can control any robot using this controller. When an operator uses a mouse to click on the robot in the diplay and then clicks an point where he/she wants to observe, the sink controller generates a control packet and sends it to the selected robot. Some intermediary robots forward this packet to the destination robot. Thus, the operator can control any robot by using the sink controller.

3 Self-deployment with Connection Priority Criteria 3.1 Design Concept Robots explore the target and maintain a communication link with the sink. Robots reconstruct a bi-directional multihop communication route with movement. In some instances, robots move according to the command of an operator at the sink. A selfdeployment algorithm is a collaborative system to realize these abilities.

3.2 Simulation Environment The simulation field is shown in (Fig. 2). The sink is the data gathering node of the network, from which an operator monitors the robot formation and observes the data robot sensing. The target is a point that the operator attends to observe. Each robot has sensing, computation, communication, and locomotion capabilities. The communication range of a robot is R[m], and the sensing range of a robot is

54

J. Takahashi et al.

a

v=L [m/s]

200*L

y (0, 0)

Target

Sink

Sink

c

e

b

R

f Sink

d

x

b

c a

i

i

Target

Target

200*L

(1)

(2)

(3)

Fig. 2 Simulation display and schematic diagram of the mobile sensor network

Rs [m]. It is assumed that each robot knows its own position, as well as the positions of the sink and the target. This assumption is valid because any acquired information has to be related to the positional data as well as time information. Although the localization of a robot is important technique, it should be treated in a different context. The mission of the robot is to observe an arbitrary point in an environment and send sensing data to the sink. When the target is out of the communication range, robots must move in a manner that will maintain the communication link (Fig. 2(3)).

3.3 Definition of Symbols This section defines the symbols used for algorithm formulation (Fig. 3). Each robot ID is represented as a number (i = 1, 2, 3, · · · ). In particular, if the node is a sink i = s, a target is treated like a node and i = t. The position of node, the sink, and the target are described, respectively, as follows: P i (xi , yi ), Ps (xs , ys ), Pt (xt , yt ).

(1)

The distance between robot i and robot j is defined as follows: D(Pi , Pj ) = ||Pi − Pj ||

[m].

(2)

The set of robots existing in the communication range (R[m]) of robot i is described by Ni and is defined as follows: Ni = {k|D(Pi , Pk ) < R}.

(3)

The robots that are closer to the sink than robot i are described by Nsi ⊂ Ni and are defined as follows: Nsi = {k|D(k, s) < D(i, s), k ∈ Ni }.

(4)

Cooperative Object Tracking with Mobile Robotic Sensor Network

N3s area

Behavior architecture

7

Decision of connection priority

6

Calculate relative distance

3 Sink

5 2

4

1

Decision of heading direction Generate virtual force

+

+

Detect obstacle

Receive beacons

Target

55

Sonar sensing

Drive motor

N3t area Robots

Fig. 3 Particular case of robot 3: N 3 = {2, 4, 5, 6}, Ns3 = {4, 5, 6}, Nt3 = {2, 4}, First Priority = robot 5, Second Priority = robot 2

Environment

Fig. 4 Conceptual diagram of behavior architecture based on connection priority criteria with obstacle avoidance

The robots that are closer to the target than robot i are described by Nit ⊂ Ni and are defined as follows: Nit = {k|D(k, t) < D(i, t), k ∈ Ni }.

(5)

Using these symbols, the desired condition of the network is defined as follows: 1.

At least one robot (i = 1, 2, · · · , n) must move to the target so that it can observe the target. D(Pt , Pi ) < Rs

2.

(6)

A connection link from robot i to the sink exists. (Fig. 2(3) i, a, b = 1, 2, · · · , n) Ni = {a}, Na = {i, b}, · · · , Nf = {e, s}

(7)

3.4 Robot Movement Criteria 3.4.1

Algorithm Overview

A diagram of the concept of the algorithm architecture is shown in Fig. 4. First, the robot receives beacons from neighbor robots and so has knowledge of the information of the neighbors. Second, the robot calculates the distances between robots, the sink, and the target. Third, the robot decides the connection priority with a neighbor robot. Finally, the robot moves to a position that allows a stable communication link with a higher priority robot to be maintained.

56

3.4.2

J. Takahashi et al.

Decision of Connection Priority

In the case of Fig. 3, robot 3 decides the connection priority to its neighbors based on the following criteria using relative distance information between neighbors, the sink, and the target. Let the highest priority robot and second highest priority robot for robot 3 be First Priority and Second Priority. First Priority: The robot closest to robot 3 in N3s . When N3s = φ, the First Priority is the sink. Second Priority: The robot closest to robot d in N3t . When N3t = φ, the Second Priority is the target. Priority rule: One robot can only be assigned one priority. In Fig. 3, the First Priority is robot 5 and the Second Priority is robot 2. 3.4.3

Decision of direction

Next, each robot moves based on connection priority. When the distance between any two robots is less than Rs t = α · R[m](0 < α < 1), they have a stable communication link. Movement rule I: Robot d moves to a position so as to maintain stable communication link with the First Priority robot. If this is not possible, robot d moves to the position nearest to the First Priority robot within the movable range. Movement rule II: As long as the stable connection to the First Priority robot is maintained, robot d moves to a position so as to maintain a stable communication link to the Second Priority robot. If this is not possible, robot d moves to the position nearest to the First Priority robot within the movable range. 3.4.4

Obstacle Avoidance

For obstacle avoidance, based on subsumption architecture, a Virtual Force Field method[9], which is a simple algorithm that uses a mobile robot equipped with sonar, is integrated. A diagram of the concept of the algorithm integration is shown in Fig. 4. 3.4.5

Simulation

The functional capability of the self-deployment algorithm is investigated by simulation. The self-deployment process, from the initial random configuration to the desired configuration, is shown in Fig. 5. The parameter values used in the simulation are described in Table 1. The radio wave can pass through obstacle walls, whereas this is not possible for a robot. In the early stage, since Nis = φ(i = 1, 2, · · · ) for most robots, the sink is set as the First Priority, and the robots move toward the sink (Fig. 5, t = 0 ∼ 30). When a robot reaches a stable communication area of the sink, or the First Priority robot, it moves to a position so as to maintain a stable communication link with the

Cooperative Object Tracking with Mobile Robotic Sensor Network

57

Table 1 Parameter values used in the simulation Max velocity Communication range Stable communication range Sensing range Sink position Target position

Vmax R Rst Rs Ps Pt

1.0 [m/s] 25 [m] 20 [m] 0.3 [m] (60, 60) (−60, −60)

Fig. 5 Self-deployment process of mobile robotic sensor network

Second Priority robot, which has already been set, in accordance with Movement rule II (Fig. 5, t = 30 ∼ 156 (5)). Finally, the robot group achieves the desired configuration (Fig. 5, t=182). Approximately 180[s] is required for the robot group to achieve the desired condition.

4 Network Routing for Mobile Robotic Sensor Network 4.1 Ad-hoc On-Demand Distance Vector (AODV) Routing Method The purpose of AODV[10] is to build the shortest maultihop communication route between any two robots so that they can communicate with each other, even though they are separated by some distance. The route information of the two robots is stored in the intermediary robots between the two robots in question. If robot D and robot K want to communicate (Fig. 6), bi-directional communication routes are generated by the following procedure: Step 1. Robot D broadcasts the Route Request (RREQ) packet to robot K in the network. Step 2. After receiving the RREQ, a robot registers the intermediary robot as the next hop robot and forwards the RREQ. All robots that receive the RREQ of robot D have a communication route to robot D. Step 3. When robot K receives the RREQ, it sends the Route Reply (RREP) to robot D using the constructed route.

58

J. Takahashi et al.

Step 4. Receiving the RREP, a robot registers the intermediary robot as the next hop robot and forwards the RREP according to its own route information. All robots that receive the RREP of robot K know the communication route to robot K. G

G

K

H

F

F

J

D

G

K

H

D

D

J

D

D

F

D

D

K

(1)

I

E

D

D

D

D

C

D

B

A

(2)

D

D

C

D

B

F

K

H

D D

D

K

E

D

A

(3)

D

I

E

D

D

D

D

C

D

B

D

J

K

D

I D

D

D

J

K

D

E

G

K

H

D

D

A

(4)

I

K D

D

K

D

D

C

D

B

D

A

Fig. 6 Routing process of mobile sensor network

4.2 Adaptation of Robot Behavior Luckily, a portion of the self-deployment algorithm can be used as the communication route. In other words, the First Priority of each robot is equivalent to the next hop robot toward the sink. Therefore, as long as self-deployment is working, robots always have a communication route toward the sink. Thus, Step 2 of the AODV procedure is accomplished. To build a communication route from the sink to each robot, each robot runs Steps 3 and 4 of the AODV. Each robot sends a RREP (named s-beacon) to the next hop robot toward the sink, in periodic intervals. Since an s-beacon includes information such as the position of source robot, the sink controller can illustrate the robot on a display.

5 Experiment of Multihop Teleoperation 5.1 Experimental Conditions The experimental setup and the display of the sink controller are shown in Fig. 7. The parameter values used in the experiment are described in Table 2. The experimental plan is shown below. Step1. Step2. Step3. Step4. Step5. Step6.

Power on the robot to start communication Assign the position of target 1 from the sink controller Make robots begin to move When a robot reaches the target, it transmits the data pkt to the sink Assign the position of target2, after some data pkts arrived at the sink Robots begin to reforme their formation for sensing target2.

The target positions are T arget1(1, 15) and T arget2(2, 6). The sink is on the origin. There are five mobile robots, which are numbered from 4 to 8. The communication range (R) and the stable connection range (Rst ) of each robot are set to

Cooperative Object Tracking with Mobile Robotic Sensor Network

59

Table 2 Parameter values used in the experiment Max velocity of robot Communication range Stable communication range Sensing range Sink position Target1 position Target2 position

Vmax R Rst Rs Ps Pt1 Pt2

1.0 [m/s] 4.0 [m] 3.0 [m] 0.3 [m] (0, 0) (1, 15) (5, 3)

Fig. 7 Experimental setup and display of the sink controller

R = 4.0[m] and Rst = 3.0[m], respectively. In this experiment, the radio wave of the ZigBee module could be received at a point 15[m] away, and the possibility of receiving a packet is determined by software based on the distance between the robots. Since there is no localization technique, each robot knows its position by dead-reckoning. Each robot sends a beacon signal including position information every 2 seconds and an s-beacon (=RREP) for generating a multihop route to the sink every 4 seconds. A robot receiving a beacon then knows the position of neighbor robots. It then decides the connection priority and the desired direction using the position information of neighbor robots. On the other hand, when a robot receives an s-beacon, it forwards this packet to the First Priority robot and makes a route table at the same time. When a robot reaches the target, it starts sensing. However, thus far, since robots do not have any sensors, it virtually generates sine wave data every 0.1 seconds, despite real sensing. It then sends a data packet, which consists of the virtual sensing data and time to the sink, through the multihop communication route. Since the communication range is also restricted to 4[m], the sink cannot receive a beacon from a remote robot. However, the sink controller draws the robots and communication links on the drawing field (Fig. 7) by using information of an s-beacon transmitted through the multihop communication route. If intermediate robots are temporarily unable to communicate, the more distant robots are not drawn by necessity, because the s-beacons of these robots are not transmitted to the sink.

60

J. Takahashi et al.

Fig. 8 Experimental result of multihop teleoperation

When the sink controller receives the data packet, it plots the virtual sensing data on the sensing data field (Fig. 7) in accordance with the time axis. Analyzing the sensing data field graph, we can evaluate the performance of this mobile robotic sensor network from the viewpoint of stability and time delay between the data sensing time and the data receiving time. On the other hand, any robots having an available link to the sink can be operated by the sink controller. The operator can select any robot in the drawing field (Fig. 7) and have it to move to any position. The operation command packets also are transmitted to the selected robot using the constructed multihop route. In this case, other robots move to a new position to maintain the communication links between the sink and the operated robot.

5.2 Experimental Results The experimental process and the display of the sink controller are shown in Fig. 8. The process of self-deployment for building the multihop network between the sink and target 1 is confirmed from the first three pictures. In the third picture, robot 7 reaches target 1, and the sink controller displays the present position of the robots, the communication link, and the collected sensing data. From this result, the validity of the proposed self-deployment algorithm is shown. In addition, the up-current communication route was found to be built appropriately by the routing algorithm.

Cooperative Object Tracking with Mobile Robotic Sensor Network

61

When the operator orders the robots to move toward target 2 (in the fourth picture), all of the robots start to move toward target 2. This indicates that the control command packet was confirmed to be sent to each robot without packet loss. Therefore, the down-current communication route was built appropriately by the routing algorithm. The arrival rate of the data packet is approximately 50 ∼ 80%. The low arrival rate is mainly due to the collision of wireless communication based on CSMA protocol. It can improve by being utilized the advanced collision avoidance protocols for wireless mesh sensor network, such as the phase diffusion time division (PDTD) method[11].

6 Conclusions In this paper, the mobile rbotic sensor network and the sharing awareness methode are proposed for unresolve problems of coorpolative object tracking. About the mobile robotic sensor network, the self-deployment algorithm, routing algorithm based on AODV and the robot system which can communicate with Peer-ti-Pee are proposed. Fathermore, the effectivity of these algorithm and this robot system are proved by the success of the multihop teleoperation experiment. The object tracking technique exists on the extension of multihop teleoperation technique. In above experiment, the operator indicates the target point. If a robot in the network can decide the target automatically, it updates the target position according to target object movement and broadcast the new target position using the network, and other robots move to maintain multihop communication route betwee that robot and the sink. By this means, one of the object tracking can be achieved. Acknowledgements. This work is supported in part by Grant-in-Aid for Scientific Research (KAKENHI (C) 19500145) and Hori Information Science Promotion Foundation 17th.

References 1. Stillman, S.T., Tanawongsuwan, R., Essa, I.A.: A System for Tracking and Recognizing Multiple People with Multiple Camera. Georgia Institute of Technology (1998) 2. Chang, T.H., Gong, S.: Tracking multiple people with a multi-camera system. In: IEEE Workshop on Proceedings of Multi-Object Tracking, pp. 19–26 (2001) 3. Kettnaker, V., Zabih, R.: Bayesian multi-camera surveillance. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1999, vol. 2 (1999) 4. Parker, L.: Cooperative robotics for multi-target observation. Intelligent Automation and Soft Computing 5(1), 5–19 (1999) 5. Black, J., Ellis, T.: Multi camera image tracking. Image and Vision Computing 24, 1256– 1267 (2006) 6. Jung, B., Sukhatme, G.S.: Cooperative Multi-robot Target Tracking. In: Proceedings of the 8th International Symposium on Distributed Autonomous Robotie Systems, Minneapolis, pp. 81–90 (2006)

62

J. Takahashi et al.

7. Howard, A., Mataric, M.J., Sukhatme, G.S.: Mobile sensor network deployment using potential fields: A distributed, scalable solution to the area coverage problem. In: Proceedings of the 6th International Symposium on Distributed Autonomous Robotic Systems, pp. 299–308 (2002) 8. Pac, M.R., Erkmen, A.M., Erkmen, I.: Scalable Self-Deployment of Mobile Sensor Networks: A Fluid Dynamics Approach. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1446–1451 (2006) 9. Borenstein, J., Koren, Y.: Real-time obstacle avoidance for fast mobile robots in clutteredenvironments. In: Proceedings of IEEE International conference on Robotics and Automation, pp. 572–577 (1990) 10. Perkins, C., Belding-Royer, E., Das, S.: RFC3561: Ad hoc On-Demand Distance Vector (AODV) Routing. Internet RFCs (2003) 11. Sekiyama, K., Kubo, Y., Fukunaga, S., Date, M.: Distributed Time Division Pattern Formation for Wireless Communication Networks. International Journal of Distributed Sensor Networks 1, 238–304 (2005)

Deployment and Management of Wireless Sensor Network Using Mobile Robots for Gathering Environmental Information Deployment andMobile Management Network Using Robots of Wireless Sensor Tsuyoshi Suzuki , Ryuji Sugizaki, Kuniaki Kawabata , Yasushi Hada, and Yoshito Tobe T. Suzuki et al. 1

2

3

Abstract. This paper describes an information gathering system for indoor environment using a wireless sensor network (WSN) and a mobile robot, which is assumed to be applied in disaster areas. In this system, the mobile robot carries sensor nodes and deploys them onto the ground with measuring electric field strength, thereby enabling the WSN to be constructed and managed adaptively. When the WSN is disconnected by any causes, the mobile robot maintains the connection by setting additional nodes to suitable position. Utilizing proposed system, the mobile robot gathers environmental information from the WSN. The prototype system was developed and experiments were described to evaluate proposed method.

1 Introduction Large-scale earthquakes occur in various regions of the world, and the disaster awareness of most populations has risen in recent years. Thus, in addition to disaster prevention, researchers have focused on "disaster mitigation," which reduces the damage after the occurrence of a disaster. In disaster mitigation, it is necessary to understand the disaster situation as promptly as possible. For instance, analytical research conducted following the Great Hanshin-Awaji Earthquake (Kobe, Japan) of 1995 highlighted the importance of the prevention of secondary disasters such as fires occurring after the main incident to the effectiveness of the rescue Tsuyoshi Suzuki, Ryuji Sugizaki, and Yoshito Tobe Tokyo Denki University, 2-2, Kanda-Nishiki-cho, Chiyoda-ku, Tokyo 101-8457, Japan e-mail: [email protected], [email protected], [email protected] Kuniaki Kawabata RIKEN, 2-1, Hirosawa, Wako, Saitama 351-0198, Japan e-mail: [email protected] Yasushi Hada National Institute of Information and Communications Technology, 4-2-1 Nukui-Kitamachi, Koganei, Tokyo 184-8795 Japan e-mail: [email protected]

64

T. Suzuki et al.

operation. Therefore, it is crucial that information gathering and surveillance be promptly and continuously implemented in disaster areas. Study of the manual deployment of sensor nodes on building ceilings, streetlights, etc., for preemptive disaster area information gathering post-disaster[1] is in rescue research projects[2]. Preemptively deployed fixed sensor nodes can collect information at any time continuously before a disaster occurs. However, their functionality following a disaster cannot necessarily be guaranteed. If people are relied upon to deploy sensor nodes, human resources are often insufficient. Moreover, human sufferings might be caused by secondary disaster during information gathering activity.

Fig. 1 Conceptual Sketch of MRSN-based Environmental Information Gathering Support System

Based on the above mentioned information, we thought that we would be able to preserve the sensing functionality and the network through the prompt deployment of a Wireless Sensor Network (WSN) by using a mobile robot, and that we would be able to collect disaster area information adaptively. Mobile robots can be used and act as high-performance mobile sensor node in WSN by their sensing and communication capabilities. Through a fusion of WSNs and robotics, it may be possible to extend communication and sensing areas, replace failure nodes and reconstruct WSN by relocating its sensor nodes by robots. Such a system, which we refer to as MRSNs (Multi-Robot Sensor Networks), is potentially applicable as an information support system for information gathering, and situation awareness in disaster areas, etc. Figure 1 shows conceptual sketch of MRSN-based information support system for gathering post-disaster environmental information. The rest of the paper is organized as follows: Section 2 explains the autonomous construction and management of WSNs by mobile robots. Section 3 details prototype system for verification of a proposed method and Section 4 shows the experimental setup and results. Section 5 discusses future direction of our research. Concluding remarks are given in Section 6.

Deployment and Management of Wireless Sensor Network Using Mobile Robots

65

2 Environmental Adaptive Construction and Management of WSNs by Mobile Robots Based on Electric Field Strength Measurements The autonomous construction of WSNs has been previously discussed in conventional studies on sensor node deployment. The methods proposed in these studies consist mainly of randomly scattering many low-cost sensor nodes, constructing WSNs with mobile sensor nodes, etc.[3, 4, 5]. However, in the scattering deployment method, there is a possibility that the sensor nodes may not necessarily be deployed to the desired locations. Although it may be possible to evade such a problem by scattering a massive number of sensor nodes, the costs and increased communication traffic associated with the deployment of a large number of sensor nodes may prove problematic. In WSNs composed of mobile sensor nodes, the sensor nodes can be deployed to any position desired. However, in terms of mobility and adaptability, the cost-effectiveness of mobile sensor node seems low. Moreover, the sensor nodes need not necessarily all move, depending on the environment. Other methods for sensor node deployment have been proposed; these were based on maximum communication range using mobile sensor nodes or mobile robots [6, 7] and deployment by virtual interaction between sensor nodes based on physical models [8, 9]. However, these studies assumed that it would be difficult to guarantee communication between sensor nodes due to obstacles and interference waves that might block communication channels. Moreover, it is possible for that the communication between sensor nodes in WSNs to be interrupted due to decreases in sensor node battery levels or device breakage. Therefore, it is necessary to understand the status of WSNs, to ensure communication between sensor nodes, and to maintain their functionality as adaptive communication network. In our approach, robots manipulate fixed sensor nodes, and change the communication and sensing range and network topology of WSN adaptively according to the conditions of the post-disaster environment. In order to ensure communication between sensor nodes, the electric field strengths between nodes are monitored while the robot-deployed nodes are in transit to their designated locations. The robots confirm that the sensor nodes can communicate with one another, and deploy sensor nodes while guaranteeing communication channels between them. This proposed method is expected to enable construction of WSNs adaptable to changes in field strength caused by environmental interference. After such the WSN is constructed, the circumstances under which it would be unable to continue functioning are estimated according to the battery level decrease that would result in the failure of a sensor node. The robots can then specify the necessary details for a replacement sensor node by using positional information recorded when the original sensor node was deployed (odometric information, for instance). When a signal can be received from the sensor node, the accuracy of the detection of its location is improved using the field strength. Finally, the mobile robot moves to the location of the target sensor node, and the alternative sensor node is deployed. Then, the function of the WSN is maintained. In the related research, deployment and repair of the sensor network by UAV are exist[10]. However, it’s difficult for UAV to move in rubble and underground space. We assume deployment of

66

T. Suzuki et al.

WSN by mobile rescue robot systems for underground space that have been developing in disaster mitigation researches[2].

3 A Prototype System for Verification of Proposed Method A test platform for the construction of the WSN in an indoor environment was developed and tested to verify the proposed method. The omni-directional mobile robot ZEN [11] was used as the mobile robot platform (Fig. 2(a)). Mica2 MOTEs (Crossbow Technology, Inc.) were used as the sensor nodes (Fig. 2(d)). An each sensor node had a unique ID. Because the sensor nodes sent the transmission signals to on another that included the values of their own battery voltages, the condition of each of sensor node could be monitored over the WSN. A sensor node transportation and deployment device was developed for WSN construction. The device consisted of a sensor node tray (Fig. 2(c)) into which sensor nodes were placed and a sensor node manipulation mechanism (Fig. 2(b)) able to carry and place the sensor node tray on the ground. This sensor node tray moved up and down in a vertical direction along the guide due to the screw turned by Motor 2. Motor 1 moved the entire unit including screw and Motor 2 up and down vertically. The sensor node tray could be grounded by turning Motors 1 and 2. A single robot was able to transport and install 5 – 10 nodes at once using this device.

Fig. 2 Overview of prototype system

Deployment and Management of Wireless Sensor Network Using Mobile Robots

67

4 Experimental Set-Up and Results 4.1 Autonomous and Adaptive Construction and Management of WSNs by a Mobile Robot Based on Electric Field Strength A preliminary experiment was conducted in order to confirm the characteristics of the electromagnetic waves propagated between sensor nodes in order to enable stable communication between these nodes. The results confirmed that an electric field strength threshold of -70 dBm would be needed to ensure stable communication between sensor nodes. This experiment measured the electric field strength of a robot moving after installing a sensor node. The robot installed sensor node one by one measuring the electric field strength threshold of -70 dBm. The experiment was executed in an indoor passage (height: about 2.24 m, width: about 1.77 m, total length: about 40 m) of a ferroconcrete building. The scenario modeled in the experiment was as follows. The robot was given the task of installing sensor nodes. The robot initially placed a sensor node on the ground after receiving a command to carry out the autonomous construction of the WSN. The robot moved while simultaneously measuring the electric field strength between sensor nodes until it reached a preset value. The second sensor node was deployed at the point where this occurred. The robot constructed the WSN by repeating this operation. In this experiment, the battery levels of the sensor nodes were also assumed to be randomly decreasing. The robot was programmed to detect sensor nodes with low batteries, move to vicinities of such nodes, and deploy replacement sensor nodes nearby in order to maintain the WSN. Figure 3 shows an outline of the experiment based on this scenario. Photographs of the actual experiment are shown in Fig. 4. In this experiment, the robot deployed five nodes. The robot constructed the WSN using Nodes 1 to 4 by measuring electric field strength and deploying the sensor nodes with respect to its values. A simulated low battery signal was then sent from Node 1. The robot registered the status of Node 1 via the WSN and moved to a nearby location.

Fig. 3 Outline of the experiment based on the scenario

68

T. Suzuki et al.

Fig. 4 Actual Experiment on the Autonomous Adaptive Construction and Management of a WSN using a MOTE and Omni-directional Mobile Robot

A replacement sensor node, Node 5, was then deployed within the communication range of Node 2. Figure 4 (a) shows the construction of the WSN. Figure 4 (b) shows that Node 5 was deployed near Node 1 after the simulated low battery signal was detected. Figure 5 shows the distances between the successive sensor nodes. In this experimental environment, there was a distance difference of up to 10 cm between deployed sensor nodes. This is because the location of each sensor node was decided according to fluctuations in the electric field strength due to the particular characteristics of the sensor node and the status of communication within the environment. Therefore, we confirmed that it was possible to construct a WSN that ensured communication channels between sensor nodes and to manage a WSN using proposed method to restore interrupted communication paths.

4.2 Environmental Information Gathering by Using Constructed WSN To confirm the performance of the constructed WSN, illuminance data as a sample of environmental information was gathered. In this experiment, the lamp would be lit in 30 seconds after the start of experiment, and it was turned off the 30 seconds later. An each sensor node measured illuminance and transmitted data to the robot via the WSN. This experiment was carried out with the interior lights had been turned off at night. This experiment, for example, was assumed that the early stage of the fire broke out near WSN constructed and managed by mobile robot, and WSN measured fire's illuminance. Figure 6 (a) shows the experimental situation. In Fig. 6 (b), it was assumed that sensor node 1 broke down, and sensor node 5 was deployed as the alternative node. Measurement data of an each node is shown in Fig. 7. In these results, an each node could detect the environmental information changes. Figure 7 (b) showed that the performance of the WSN was able to be maintained because sensor node 5 was substituted for sensor node 1.

Deployment and Management of Wireless Sensor Network Using Mobile Robots

69

Fig. 5 Distance between Deployed Sensor Nodes

Fig. 6 Experimental Situation for Measuring Illuminance Data

5 Discussion and Future Direction The strategy in this paper will be applied for spherical sensor nodes that we are developing for gathering disaster area information (Fig. 8)[12][13]. In consideration of use in disaster area, a passive pendulum mechanism for keeping posture of sensors and an impact-resistance capability by soft outer shell is installed in these sensor nodes. We assume this system is used in underground spaces such as subway station and underground mall. Therefore, it is assumed to cover the area of about 15-30 square meters along the standard of the underground structure. In order to determine the deployment position of sensor node, not only electric field strength but also packet jitter, for example, will be measured for evaluate communication performance. Moreover, to deploy the sensor node in the place where the robot cannot get over, the throwing deployment method is discussing. In the future, these element technologies are integrated, and performance of MRSN must be further developed and improved to enable its practical application as gathering disaster information in underground space.

70

T. Suzuki et al.

(a) Illuminance Data of Each Sensor Node

(b) Illuminance Data after Deployment of Alternate Sensor Node 5

Fig. 7 Experimental Example of Illuminance Data

(a)

(b) 250[m

Linux computer, Fish eye camera, Wireless LAN, Battery, etc.

Fig. 8 (a) Sensor node with passive pendulum mechanism, (b) Soft outer shell for impact resistance capability

6 Conclusion This paper has described the autonomous and adaptive construction of WSN using MRSNs, which aim to develop support systems for information gathering in disaster areas. Section 2 proposed construction method of WSN measuring electric field strength by using mobile robot, and Section 3 described a test platform for verification of proposed method. Section 4 showed that a robot was able to construct and manage a WSN autonomously and adaptively by measuring electric

Deployment and Management of Wireless Sensor Network Using Mobile Robots

71

field strength, and also showed that a system could gather environmental information. Section 5 described our future direction. The aims include the integration and upgrade of the component technology, as well as the improvement of the system so as to enable its use within realistic environments such as the outdoors and disaster areas. In addition, we will examine the communication protocol, information management, data transfer routing and the integration and processing of a large flow of information that would be appropriate for the proposed method. Acknowledgments. The work presented here is a portion of the research project “Robot sensor network for rescue support in large-scale disasters, “supported by the Research Institute for Science and Technology, Tokyo Denki University, Japan.

References 1. Kurabayashi, D., Asama, H., Noda, K., Endo, I.: Information Assistance in Rescue using Intelligent Data Carriers. In: Proceedings of 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems, Maui, Hawaii, USA, pp. 2294–2299 (October 2001) 0-7803-7398-7 2. Tadokoro, S., Matsuno, F., Onosato, M., Asama, H.: Japan National Special Project for Earthquake Disaster Mitigation in Urban Areas. In: First International Workshop on Synthetic Simulation and Robotics to Mitigate Earthquake Disaster, Padova, Italy, pp. 1–5 (July 2003) 3. McMickell, M.B., Goodwine, B., Montestruque, L.A.: MICAbot: A Robotic Platform for Large-Scale Distributed Robotics. In: Proceedings of the 2003 IEEE International Conference on Robotics & Automation, Taipei, Taiwan, pp. 1600–1605 (September 2003) 0-7803-7736-2 4. Dantu, K., Rahimi, M., Shah, H., Babel, S., Dhariwal, A., Sukhatme, G.S.: Robomote: enabling mobility in sensor networks. In: Proceedings of Fourth International Symposium on Information Processing in Sensor Networks, UCLA, Los Angeles, California, USA, pp. 404–409 (2005) 0-7803-9201-9 5. Parker, L.E., Kannan, B., Fu, X., Tang, Y.: Heterogeneous Mobile Sensor Net Deployment Using Robot Herding and Line of Sight Fromations. In: Proceedings of the 2003 IEEWRSJ International Conference on Intelligent Robots and Systems, Las Vegas, Nevada, pp. 2488–2493 (October 2003) 0-7803-7860-1 6. Miyama, S., Imai, M., Anzai, Y.: Rescue Robot under Disaster Situation: Position Acquisition with Omni-directional Sensor. In: Proceedings of the 2003 IEEWRSJ International Conference on Intelligent Robots and Systems, Las Vegas, Nevada, pp. 3132– 3137, October 27-31 (2003) 0-7803-7860-1 7. Sugano, M., Kawazoe, T., Ohta, Y., Murata, M.: An indoor localization system using RSSI measurement of a wireless sensor network based on the ZigBee standard. In: Proceedings of the sixth IASTED International Multi-Conference on Wireless and Optical Communication, Banff, Canada, pp. 503–508 (July 2006) 8. Howard, A., Matari’c, M.J., Sukhatme, G.S.: Mobile Sensor Network Deployment using Potential Fields: A Distributed, Scalable Solution to the Area Coverage Problem. In: Proceedings of Distributed Autonomous Robot System 2002, Fukuoka, Japan (June 2002)

72

T. Suzuki et al.

9. Pac, M.R., Erkmen, A.M., Erkmen, I.: Scalable Self-Deployment of Mobile Sensor Networks: A Fluid Dynamics Approach. In: Proceedings of International Conference Intelligent Robots and Systems, Beijin, Chaina, pp. 11451–14460 (October 2006) 14244-0259-X 10. Corke, P., Hrabar, S., Peterson, R., Rus, D., Saripalli, S., Sukhatme, G.: Autonomous Deployment and Repair of a Sensor Network using an Unmanned Aerial Vehicle. In: Proceeding of IEEE International Conference on Robotics and Automation, New Orleans, LA, pp. 3602–3608 (October 2004) 0-7803-8232-3/04 11. Asama, H.: Development of an Omni-Directional Mobile Robot with 3 DOF Decoupling Drive Mechanism. In: Proceedings of IEEE International Conference on Robotics and Automation, pp. 1925–1930 (1995) 0-7803-1965-6 12. Sato, H., Kawabata, K., Suzuki, T., Kaetsu, H., Hada, Y., Tobe, Y.: Information Gathering by wireless camera node with Passive Pendulum Mechanism. In: Proceedings of International Conference on Control, Automation and Systems 2008, Seoul, Korea, October 14-17 (accept, 2008) 13. Sawai, K., Suzuki, T., Kono, H., Hada, Y., Kawabata, K.: Development of a Sensor Node with Impact-Resistance Capability to Gather the Disaster Area Information. In: 2008 International Symposium on Nonlinear Theory and its Applications NOLTA 2008, Budapest, Hungary, pp. 17–20, September 7-10 (2008)

Global Pose Estimation of Multiple Cameras with Particle Filters⋆ Ryuichi Ueda, Stefanos Nikolaidis, Akinobu Hayashi, and Tamio Arai

Abstract. Though image processing algorithms are sophisticated and provided as software libraries, it is still difficult to assure that complicated programs can work in various situations. In this paper, we propose a novel global pose estimation method for network cameras to actualize auto-calibration. This method uses native information from images. The sets of partial information are integrated with particle filters. Though some kinds of limitation still exist in the method, we can verify that the particle filters can deal with the nonlinearity of estimation with the experiment.

1 Introduction Systems that contain many cameras have been utilized in various situations for decades. In recent years, especially, such a multiple camera system becomes close-to-home by appearance of network cameras, which can be connected to IP-based networks. In this paper, we propose a novel method that estimates relative poses of some monitoring cameras that are installed in a room environment. The difficulty of this problem is that the cameras are not always rigidly fixed in an environment. In the case of two cameras that are fixed for stereophonic vision, the method for calibration is established. Data association problems can be easily solved by cutting-edge methods in OpenCV [1], and the positions of corresponding points in the images bring internal and external parameters of the cameras [2, 3]. In the case of distributed cameras, however, its difficult to distinguish whether images from two cameras contain a common part. It is difficult to Ryu i c h i U e d a, S t e f an os N i k ol ai d i s , A k i n ob u H ay as h i , an d T am i o A r ai D e p t . of P r e c i s i on E n gi n e e r i n g, U n i v . of T ok y o, 7- 3- 1 H on go, B u n k y o- k u , T ok y o e - m ai l : {ueda,stefanos,akinobu,arai-tamio}@robot.t.u-tokyo.ac.jp ⋆

T h i s r e s e ar c h i s p ar t i al l y s u p p or t e d b y t h e M i n i s t r y of E d u c at i on , S c i e n c e , S p or t s an d C u l t u r e , G r an t - i n - A i d f or Y ou n g S c i e n t i s t (B) (No. 19760163).

74

R. Ueda et al.

set a threshold for this judgement and errors always happen. The risk of occlusion enhances. Resultingly, their poses can be measured robustly only from simple information. Moreover, each relative pose of every pair of cameras should be calculated without contradiction. Though we can think that internal parameters of network cameras are calibrated by their vendor, it will be not precise. Therefore, we need a method that roughly and robustly estimates external parameters of each camera. After rough estimations, the precise calibration method can be done by data association. The proposed method deals with these problems by utilization of particle filters. This method integrates partial information into poses of cameras gradually through observations of a moving robot. Particle filters can handle nonlinearity of this problem with straightforward algorithms. The proposed method also provides a modification procedure for contradictions between estimation results.

2 Pose Estimation of Multiple Cameras A camera system is mathematically defined in this section. As shown in Fig.1, we assume an environment that has cameras {Cami |i = 1, 2, . . . , n}. Since the elements of the proposed method can be explained with three cameras, only three cameras: Cam1 , Cam2 , Cam3 are considered in this paper for simplicity. A Cartesian coordinate system Σi is related to each Cami (i = 1, 2, 3). Though there is no requirement, the z-axis of each coordinate system is identical with the optical axis of each camera. We assume that every pair of the cameras have an intersectional part in their fields of view. There is an object that can be observed from the cameras continually in the environment. The 3-dimensional (3D) position of this object is represented as i pobj in the coordinate system Σi . When Cami is observing the object, measurements of the position are obtained continually. A measurement by the computer that obtains an image from

Fig. 1 Assumed Camera System

Fig. 2 Flow of Whole Algorithm

Global Pose Estimation of Multiple Cameras with Particle Filters

75

Cami is represented as i ζ = (i mx i my i mz )T in Σi . The difference |i pobj −i ζ| means a measurement error. We assume that all of the three values: i mx ,i my and i mz can be measured with some extent of errors. If the object is a point, only two variables can be known in general. In the case where the size of the object is known, or in the case where the object moves in a flat floor, three variables can be measured. However, this assumption is too strong and it should be removed in the next version. In this paper, the robustness of the proposed against i mz is discussed. We want to solve the poses of all cameras. In this paper, relative poses between Cam1 , Cam2 , Cam3 are estimated. The relation between two camera coordinate systems Σi , Σj (∀i, ∀j) is represented by the following homogeneous transform matrix:  i Rj i pj i , Tj = 0 1 where i Rj and i pj denote a 3 × 3 rotation matrix and a translation vector from Σj to Σi respectively. Positions of the object in Σi and Σj fulfill   i j pobj pobj . (1) =i Tj 1 1

3 Estimation with Particle Filters In this section, we propose a novel method that estimates the homogeneous transform matrices robustly. We show beforehand the flow of the proposed method in Fig.2. This method receives a pair of measurements from two cameras in sequence. The measurements are used for the pose estimation in the following procedures: sampling on measurement and filtering on measurement. In the flow, we can see the geometric sampling procedure, which is one of the most important features in this method.

3.1 Particles A particle in a particle filter is composed of a point estimate and its weight. When a particle filter estimates a homogeneous transform matrix from Camj to Cami , particles in the particle filter can be defined as [n]

[n]

[n]

tij = (i Tj , wij ) (n = 1, 2, . . . , N ), [n]

where the homogeneous transform matrix i Tj [n]

is the point estimate in six

dimensional parameter space and wij is the weight respectively. The set of the particles is represented as Tij in this paper. Tij can be changed to Tji when every particle in Tij is transformed as

76

R. Ueda et al.

Fig. 3 Relations used by the Proposed Method

j

[n]

Ti

Fig. 4 Sampling based on Measurements by Two Cameras

=



i

[n]

Tj

−1

[n]

[n]

, wji = wij .

Since a homogeneous transform matrix is represented by six parameters, we can assume a six-dimensional space where particles distribute. This space is frequently called the 6D space hereafter.

3.2 Information for Estimation In Fig.3, some links are written between the object and cameras, which are also shown in Fig.1. Two cycles can be also seen in the figure. Two kinds of information for estimation can be obtained from them. The cycle 1 indicates that information can be obtained by simultaneous measurements of the object from two cameras for estimating a homogeneous transform matrix. This information is based on the following equation: i

ζ =i Tjj ζ

(i = 1, 2, 3; j = 1, 2, 3; i = j)

(2)

if the measurements are accurate. Since this information is partial, some measurements of the object in different points are required for solving i Tj . The cycle 2 implies the following geometric constraint: I =1 T22 T33 T1 .

(3)

This relation can be used when the number of cameras are more than two.

3.3 Algorithms A: Sampling on Measurements. Here we consider the case where two cameras: Cami and Camj measure the position of the object simultaneously. The measurements are i ζ and j ζ respectively.

Global Pose Estimation of Multiple Cameras with Particle Filters

77

The first step of the proposed method is to sample possible matrices [n] Tj (n = 1, 2, . . . , N ) in the 6D space by using the measurements. In the sampling, measurement errors in i ζ and j ζ is considered. The matrices must be sampled fairly from a possible set of matrices that fulfill Eq.(2). Figure 4(a) illustrates errors on measurements. As shown in this figure, the object is usually observed with errors. We put the error model into a probability distribution p(i ζ|i pobj ) (i = 1, 2, 3). In many cases, this distribution can be decomposed into a distribution of the measurement error on the position of an image plane and that of the distance. When the optical axis of Cami is identical to z-axis of Σi , this decomposition can be represented as   i 

 mx i my  i x i y i m2 +i m2 +i m2  i x2 +i y 2 +i z 2 . p(i ζ|i pobj ) = p i , p ,i i i x y z mz mz z z (4) i

The two distributions in Eq.(4) are assumed as Gaussians in this paper. On the image plane (xy-plane), we assume that the distance between (i x/i z,i y/i z) and (i mx /i mz ,i my /i mz ) is distributed with a standard deviation σimage . The distribution of distance errors is represented by σdist . The first step of sampling is to obtain random values from the measurement model: Eq.(4). When i ζ and j ζ are obtained, a pair of procedures: i [n]

p

∼ p(i pobj |i ζ) and

j [n]

p

∼ p(j pobj |j ζ)

are executed, where ∼ denotes a sampling based on the distribution of the right side. p(j pobj |j ζ) has the identical form with p(j ζ|j pobj ). We assume that sampled positions: i p[n] and jp[n] fulfill Eq.(1) as i

 j [n]   [n] [n]  j [n]  i p p[n] p Rj i pj [n] . = =i Tj 1 1 1 0 1

This equation contains the following important relation: [n]

|i p[n] −i pj | = |j p[n] |. [n]

From the above equation, a translation vector i pj can be sampled randomly. i [n] pj

possibly denotes the position of Camj in Σi . Figure 4(b) illustrates this sampling intuitively. [n] Next we must choose a rotation matrix i Rj . This matrix possibly denotes the direction of Camj in Σi . The procedure for this sampling is as follows. This procedure is composed of tens of geometric calculations. 1. A coordinate system Σtmp is considered for convenience. The origin of [n] Σtmp is at i pj in Σi . And its z-axis points i p as shown in Fig.4(b).

78

R. Ueda et al.

Fig. 5 Geometric Sampling

2. z-axis of Σtmp is rotated randomly. 3. The directions of x-axis and y-axis are adjusted as the measurement result from Camj becomes j p[n] . [n]

The above procedure is iterated N times, i Tj

(n = 1, 2, . . . , N ) are sam[n]

pled. We can then obtain a set of particles: Tij = {tij |n = 1, 2, . . . , N }. The weight

[n] tij

of every particle is set to 1/N if there is no other information.

B: Filtering on Measurement. For further estimation, the proposed method utilizes a filtering based on Bayes rule for further estimation. We consider a case where i ζ and jζ are obtained again. The filtering process changes the weight of every particle in Tij with the following equation: [n]

[n]

wij = ηp(i ζ,j ζ| i Tj ), where η is a value for normalization. This probability can be interpreted as [n]

[n] j

wij = ηp(i ζ| i Tj

[n]

[n] j

ζ) or wij = ηp(i Tj

ζ|i ζ)

with the measurement model p(i ζ|i pobj ). We use the right side of the above equations in the implementation. Each particle filter eludes a bias of weight by using a resampling algorithm after filtering [4]. Moreover, we add an algorithm that slightly perturbs matrices of a part of particles. This algorithm relaxes concentrations of particles after every resampling process. C: Geometric Sampling. We add another sampling procedure to the proposed method though estimation of a homogeneous transform matrix between two cameras is possible by the above procedures. The additional procedure can remedy a problem pertaining to particle filters. The problem is that particles frequently converge at a wrong place in the space of estimation. This problem has been discussed in the context of mobile robot localization [5, 6]. Fortunately, the cycle 2 in Fig.3 becomes a key for this problem. In the geometric sampling, which is proposed here, some particles are replaced based on this cycle. The procedure of geometric sampling is illustrated in Fig.5. At first, two particles are chosen randomly from each set of particles: Tik and

Global Pose Estimation of Multiple Cameras with Particle Filters [n′ ]

[n′′ ]

Tkj . Secondly, the particles: tik and tkj [n] tij

79

are used for a choice of a particle

whose homogeneous transform matrix is i

[n]

Tj

[n′ ] k

= i Tk

[n′′ ]

Tj

.

This equation is based on Eq.(3). Finally, a particle in Tij is chosen randomly and replaced by this particle.

4 Evaluation with Simulated Measurements We investigate robustness of the proposed method with noisy measurements that are generated by a simulator. The conditions of simulation are as follows. • There are three pan/tilt cameras in a room shown in Fig.6. When a coordinate system Σ in Fig.6 is considered, positions of the base of the cameras are explained by the (x, y, z) coordinates shown in the figure. • Each camera observes a moving robot. Its position i ζ can be measured from Cami . The distance from the robot to a camera can be stochastically calculated with the width of the robot in an image. The position of the robot is changed randomly every time step. A pair of cameras observes the robot simultaneously. The cameras are chosen randomly. We consider three particle sets: T12 , T23 , T31 . Each of them includes 10000 particles. After a filtering on measurement, 1[%] of the particles in each set are slightly perturbed. The position and orientation is moved at most 0.05[m] and 5[deg] respectively. In the geometric sampling, 1[%] particles are resampled. We variously change the value of σdist and σimage , which are the parameters of noises. In this simulation, these standard deviations are used for adding to noise to simulated measurements, and also used as the parameters of each particle filter. [n] When Σi is fixed, coordinate systems: Σj (n = 1, 2, . . . , 10000) can be [n]

written from each particles in Tij . Estimations Σj and the actual Σj are compared for evaluation of the estimation. To evaluate the error, we use the following values: 1) the distance between the origin of Σj and the average [n] position of origins of Σj and 2) the average difference of angle between the [n]

z-axis of Σj and that of each Σj . They represent the transition error and the rotational error respectively. The accuracy and robustness of the particle filters are evaluated here with various extents of noise. The estimation results after 1000 steps are compared in Table 1. Values in this table are average results of three particle filters. As shown in this table, relation between error of distance and estimation error is remarkable. This problem is a well-known problem on calibration of external parameters. We need an additional algorithm in the proposed method to solve this problem in this stage.

80

Fig. 6 Simulation Environment

R. Ueda et al.

Fig. 7 Roomba Red made by iRobot and Its Recognition

Table 1 Estimation Errors after 1000[step] σ dist [m] 0.1 0.1 0.1 0.2 0.2 0.2 0.4 0.4 0.4 1.0 1.0 1.0 σ image 0.05 0.1 0.2 0.05 0.1 0.2 0.05 0.1 0.2 0.05 0.1 0.2 positional error [mm] 55 119 444 164 164 256 635 522 329 3153 2444 1915 rotational error [deg] 0.9 2.8 7.3 3.1 2.6 6.6 14.7 10.4 6.6 69.6 50.3 43.6

5 Evaluation with Actual Measurements In the experiment, a red Roomba shown in Fig.7(a) is measured from three cameras. We have used three pan/tilt cameras (SNC-RZ50N made by Sony) fixed on the ceiling of a room. We define camera coordinate systems Σi (i = 1, 2, 3), whose z-axes are perpendicular and directed to the floor. The locations of Σ2 and Σ3 in Σ1 are shown in Fig.8. We take 12 trials. In a trial, the Roomba runs the environment for 10[min]. Figure 8 shows two examples of the path observed from cameras. Every camera tracks the Roomba and measures its position. When a pair of the cameras can measure the Roomba simultaneously, the values of the measurements are recorded. 1, 400 pairs of measurements are obtained per one trial. Figure 7(b) shows a recognition result of the Roomba from one of the cameras. In the image processing for this paper, differently from the simulation,

Global Pose Estimation of Multiple Cameras with Particle Filters

Fig. 8 Positions of Roomba Observed in 10 Minute Trials

81

Fig. 9 Averaged Progression of Estimation 0.5 0.4 0.3 0.2 0.1 0.0 8

6

4

2

0

Fig. 10 Progression of Estimation on a Trial

Fig. 11 Progression of Estimation without the Geometry Sampling

the height of the room is used for calculating the position of the Roomba. The value of z of every measurement is fixed to 2.44[m]. The root-mean-square (r.m.s.) error of all position measurements in the 12 trials is 70[mm]. We use looser values for the measurement model for rough but robust estimation. The values are σdist = 0.2[m] and σimage = 0.05. Though z values of the measurements are fixed, we never use this information. In each trial, statistics of T12 , T23 , T31 are recorded at every input of a measurement set. In Fig.9, we show the total result of 12 trials, which contain 36 estimations. Each point and the vertical bar indicate r.m.s. error and the one sigma range of the errors in 36 estimations respectively. On the average, the estimations are improved monotonically in the first few minutes. Figure 10 shows the process of estimation errors in a trial. Since the difference of height between each camera and the robot is known, particles converge

82

R. Ueda et al.

near actual parameters only one minute after the robot starts moving. When the geometric sampling is not used, however, we cannot obtain meaningful result as shown in Fig.11.

6 Conclusion We have proposed a novel method for estimating the poses of three or more numbers of cameras. Each camera observes only one object, and measures the position of the object. The measurements are reflected to the particle filters, which estimate relative poses of pairs of cameras. The distributions of particle filters are amended through the geometric sampling. Through this sampling algorithm, robust convergence can be realized. We have evaluated the proposed methods with the measurements of an object in a simulation and in the actual world. In the 12 trials, the relative poses of each pair of cameras are estimated with 0.18[m] and 2.1[deg] error.

References 1. Bradski, G., Kaehler, A. (eds.): Learning OpenCV: Computer Vision With the Opencv Library. SOreilly (2008) 2. Tsai, R.: A versatile camera calibration technique for high accuracy 3D machine vision metrology using off-the-shelf TV camera and lenses. IEEE Journal of Robotics and Automation 3(4), 323–344 (1987) 3. Horn, B.K.P.: Closed-form solution of absolute orientation using unit quaternions. Optical Society of America A 4(4), 629–642 (1987) 4. Thrun, S., Burgard, W., Fox, D.: Probabilistic ROBOTICS. MIT Press, Cambridge (2005) 5. Ueda, R., et al.: Expansion Resetting for Recovery from Fatal Error in Monte Carlo Localization – Comparison with Sensor Resetting Methods. In: Proc. of IROS, pp. 2481–2486 (2004) 6. Lenser, S., Veloso, M.: Sensor resetting localization for poorly modelled robots. In: Proc. of IEEE ICRA, pp. 1225–1232 (2000)

Part II

Mobiligence

Motion Control of Dense Robot Colony Using Thermodynamics Antonio D’Angelo, Tetsuro Funato, and Enrico Pagello

Abstract. In the last decades the theory of the complex dynamical systems has come to maturation providing a lot of important results in the field of many applied sciences. Also robotics has taken advantages from this new approach in what the behavior-based paradigm is particularly suitable to devise specific sensing activity since sensors usually provide information about the environment in a form which depends on the physics of the interaction. It is not required to be immediately converted into some symbolic representation but, on the contrary, it can be maintained at some physical level as a metaphor of the events observed in the environment. The close connection between the motor schema with its companion perceptual schema seems suggesting the presence of a substratum which underlies both perception and action activities, driving the flow of information accordingly. In the paper we consider a colony of robots immersed in a well-specified thermodinamical substratum where enthalpy and heat flux are devised to go vern the diffusion/merging behavior of a swarm.

1 Introduction Sensor information can be maintained at some physical level as a metaphor of the events observed in the environment. For example, when the task of navigation is considered, a well-suited metaphor is the interpretation of the Antonio D’Angelo Dept. of Math. and Computer Science, Univ. of Udine, Italy e-mail: [email protected] Tetsuro Funato Dept. of Mecanical Engineering and Sicence, Kyoto Univ., Japan e-mail: [email protected] Enrico Pagello Dept. of Engineering and Computer Science, Univ. of Padua, Italy e-mail: [email protected]

86

A. D’Angelo et al.

motion as it comes from the particle streaming in a fluid. In this case, robot moves as a passive object along a streamline but, one can also understand its motion in an active fashion assuming objects to be electrically charged particles as the robot, but with opposite sign, so that the electrical field becomes the motion engine. Of course, neither streaming nor electrical charges are really found in the environment, but they are good metaphors to control robot motion. Basically, within this point of view, one must identify some physical properties to be assigned to the objects recognized in the enviroment so that they are easily dealt with in order to drive the task execution and completion of the robot. For example, the electrical metaphor is very popular and it is commonly used in many motion planning applications, i.e. Latombe [8], LaValle [9]; also Arkin [1] has used this metaphor to implement behaviors as a couple of perceptual and motor schemas. However, in Arkin the main motivation is to have a form of cooperation among behaviors by vector summation of motor schema output. In his change of perspective, the Brooks’ approach [4] to the robot control appears innovative in what he considers sensing a source of information ”as is”, with the environment state only partially accessible. This means that, generally, an agent cannot have a complete control on a whatsoever event in the environment: it must give up to operate in a fully deliberative mode because either the event is not recognizable or some of its properties are beyond the limit of its sensing apparatus. Moreover, natural agent behaviors show a tight connection between a motor mechanism and the perceptual paradigm through which it is governed. They exhibit a character of necessity which is responsible of coupling the motor schema with its companion perceptual schema, in Arkin’s sense, as it were ”biologically” determined. This circumstance seems suggesting the presence of a substratum which underlies both perception and action activities, driving the flow of information accordingly. Abstracting this point of view we can say the substratum is the semantic of the interaction or, more evocatively, it denotes the symbol grounding on which the robot governor’s unit operates. More recently Asama [2] has coined the term mobiligence referred to the intelligence which emerges through the interaction between an agent and its environment due to its mobility. In its seminal paper Asama supposes that such an intelligence stems from three well identifiable properties which he calls embodied plasticity, abduction and co-embodiment with the environment. Within this approach he suggests to take into account both the behavior generated by perception and also the behavior generating knowledge which become the external and internal processing of flow of information through which agents can reason about1 . But agents must be adaptive and this property requires the mapping from observation to action spaces to be augmented by featuring new abductive 1

The cognitive structure of the agent.

Motion Control of Dense Robot Colony Using Thermodynamics

go away Robot 2

Robot 1

(a) no sensor

87

Contact Sensed Follow Robot Robot Robot 2 1

(b) with sensor

Fig. 1 Sensor input

information inside the cognitive structure of the agent. To this aim an agent could be designed with an intrinsic ability to devise an internal model to cope with unknown environments only on the basis of the interaction properties. This means that the specific model of the environment is effectively generated by local information but, because the model is only needed to generate proper motor functions, such a model is not really necessary since it can be built in what we have called the substratum. Similar ideas can be applied to a colony of robot individuals performing a collective task such a swarm maintaining a group formation. Here, the main role of sensors is monitoring the task execution by suppling the right parameters to eventually adjust the relative position of the required individuals. Their evaluations depend on the increasing/decreasing noise so that it can be associated to the thermodynamic concept of entropy. Thus, if we put together several robots close each other inside a limited operating space, after a short interval of time following a perturbation to their nominal behavior, we could observe the robots to distribute over the whole working space with a significant not null probability to change again their positions. The observed whole behavior can be interpreted as gas diffusion process which, for its own dynamics, requires a suitable sensor handling to maintain the formation properties of the group. For example, the diffusion of the robot group is preserved by making robots able to react to external stimuli. In the situation depicted in fig.1(a), if sensors are available, a robot changes direction to follow other robots by using the collision stimulus as an input signal (fig.1(b)). So, what we observe is a number of robots moving towards a region in the same fashion as a spreading gas and the underlying substratum is a gas diffusion process which can be modelled through a thermodynamic paradigm. Jantz et al.[13] and Kerr et al.[14] used the kinetic theory of gases for analysing collision and realising sweep motion in a bounded region, respectively. Moreover, Kinoshita et al.[10] observed the thermodynamic parameters of a robot group to distinguish the macroscopic state of robots: random motion, periodic motion and deadlock. In such a case the substratum is more properly the kinetic theory of gas. In the present paper, we use a thermodynamic approach in order to formulate the relationship between the effect of the behaviour of a single robot and both its diffusion and merging, as it appears at macroscopic level. Each

88

A. D’Angelo et al.

Fig. 2 Robot group

robot is interpreted as a gas particle, and the energy exchange is considered through thermodynamic equations (sect. 2). As a result of this approach, we specify the conditions to handle the diffusion, and we propose a robot model to control its macro-properties as a result of its behaviour. This method, termed reaction releasing, is illustrated in sect. 3. We also emphasises the relationship between sensors and motors by applying the properties of the thermodynamic substratum to the roboticle model [5, 6].

2 Thermodynamical Metaphor The use of thermodynamical concepts is very common in applied sciences. For example, in our previous works [7] we have developed a model of autonomous robot control, called roboticle, based on the first law of thermodynamics. In our formulation it takes the form  δP = dU + δL (1) δQ = dF + V ds where δP is the thermal unordered energy entering the robot’s governor unit as perceptual flow, V ds is the mechanical ordered work driving effectors as effort. More precisely, the roboticle model makes use of two such equations, the former referred to its sensors and the latter to its effectors. Moreover, the committing effort δL, a kind of internal work, and the committed perception δQ, similar to an internal heat, are two fictitious quantities which refer to the adaptivity of the robot. The resulting autopoietic loop [7] makes use of the two scalar quantities U and F which explicitly control robot operations by estimating its task completion.

2.1 Dense Colony of Robots Let us consider a very simple model of robot swarm, where each individual has no active sensor and its motion is driven by changing direction only due

Motion Control of Dense Robot Colony Using Thermodynamics

89

to contact with other individuals. The colony of robots is placed on a arena densely enough of individuals as depicted by fig. 2. Let us regard the outermost robots as swarm boundary so that we can focus our attention on the internal dynamics of the colony itself. Instead of considering the colony as a purely reactive system, let us assume each individual to move in a brownian motion fashion, so that we could assimilate the group to gas particles moving around in a 2-dimensional space or, more properly, in a fluid with some thermal property. In such a case the area V occupied by the robots and the pressure p caused by the collision among them can be used to evaluate the global behaviour of the robot swarm. In the kinetics theory of gas the absolute temperature T is a measure of the average kinetics energy for each gas particle so that the total thermal energy Q, at the equilibrium, is due to a pure brownian motion; on the contrary, if we assume the distribution of heat inside the gas is not uniform, then a diffusion process generates an ordered motion of particles which tries to bring the gas towards a new equilibrium state. In the thermodynamic metaphor we shall assume an heating process which triggers the fluid substratum of the operating environment where the robot swarm is displaced. It generalizes the purely mechanical fluid-fulfilled substratum in what it considers the thermal activity the effective motion engine: fluid streamlines are generated by convective motion due to heat diffusion inside the fluid itself. To enter some details, let us consider the first law of thermodynamics; it requires that any variation of system heat energy δQ results in the change of its internal energy dU also providing the work δW , accordingly to the follow equation δQ = δW + dU (2) remembering that the work δW is due to the change of volume V as it stems from the well-known formula δW = pdV whereas the heat change δQ, if reversible transformations are considered, can be evaluated by δQ = T dS through entropy change estimation.

2.2 Gibbs Free Energy The diffusion process depends on the not uniform distribution of heat inside the fluid and this circumstance risults in a thermal gradient which is responsible of a massive streamline generation and each of them is the motion engine for a robot positioned on it. This substratum is governed by the heat distribution which acts on the fluid streamlines as a vector potential, in the same fashion the magnetic vector potential acts on the magnetic field. Instead of considering directly the internal energy U of the system, in the following discussion we shall refer to the enthalpy2 Heand Gibbs function3 2 3

Also, termed total heat. Also termed free energy.

90

A. D’Angelo et al.

G. Because these quantities are bound by the well-known relation G = U + pV − T S, where U + pV denotes the system enthalpy, eq. (2) yields to dG = V dp − SdT

(3)

and the Gibbs function appears the most appropriate quantity to describe the internal activity of the system. In fact, when the robot swarm is not at the equilibrium state, its internal energy U4 doesn’t change over the time since the diffusion process acts on the individual components of the group. They are forced to move accordingly to the free energy distribution inside the system which, on the contrary, keeps track of the interaction among individuals. This idea has been borrowed from chemical thermodynamics where the notion of heat of reaction is introduced to estimate the amount of energy available during the reaction and G is defined in terms of the chemical potentials µi of the reacting component substances. In the context of swarm robotic we assume the free energy to be a scalar function, defined at any point of a given region of space where the swarm is operating, with the aim to model the internal forces of the swarm due to individual interactions.

2.3 Heat Diffusion Equation In the following discussion we shall see that the diffusion process of a robot swarm can be generated by a potential field which provides an enthalpy scalar potential and a vector potential, representing the heat density of the substratum and providing the basis for the free energy G of the swarm5 . To this aim we shall start from eq. (3) by considering the rate of change of its differential terms along the time. Inside the volume V0 6 the swarm is spreading with entropy value S0 as a measure of its disorder, so that we can write dG dp dT = V0 − S0 dt dt dt accordingly to eq. (3) and the rates of change are evaluated along all the possible trajectories covered by each individual robot. If we denote with V the velocity taken along these trajectories, the total time derivatives can be immediately assimilated to material derivatives7 eventually summed up with partial time derivatives. Then, by substituting any term for the scalar product involving the gradient of the corresponding scalar quantity with the velocity itself and the partial time derivative, the preceding relation yields to 4 5 6 7

Which plays the same role of the total energy in a closed mechanical system. And, then, of the substratum. We assume a fixed region of space where the swarm is operating. Also termed convective derivative in the fluid dynamic context.

Motion Control of Dense Robot Colony Using Thermodynamics

91

∂G ∂p ∂T + V · grad G = V0 + V0 V · grad p − S0 − S0 V · grad T ∂t ∂t ∂t namely, remembering that the relation U = G − V0 p + S0 T holds, ∂U + V · [grad G − V0 grad p + S0 grad T] = 0 ∂t and also dU + V · [grad G − grad (U + V0 p) + S0 grad T] = 0 dt because the total time derivative of U requires its convective component. But, since we have assumed the robot swarm to operate without exchanging its internal energy with the outside environment so that the total time derivative of U results in the null value, the preceding relation yields to S0 ∂H = grad G − grad He ke ∂t

(4)

where we have introduced both the enthalpy He = U + V0 p and the heat flux H, which represents the rate of heat diffusion through the volume-delimiting surface, In the preceding relation we have also used the Fourier’s law of conduction which binds the temperature gradient T to the rate of heat flux H per unit cross-sectional area −ke grad T =

∂H ∂t

(5)

with ke denoting the thermal conductivity of the surface through which heat is propagating.

2.4 Substratum Properties Until now, we have considered the Gibbs function G and the heat density H as independent quantities. However, in the following discussion we depict the thermal substratum so that the total energy of the swarm is given by the heat flow through the surface σ containing the swarm itself, namely,   ˆ dσ G dV = H·n (6) V0

σ

Thus, by the divergence theorem, we can conclude that, within this model, the Gibbs function of the swarm can be taken as the divergence of the heat flux H, accordingly to G = divH so that eq. (4) becomes

92

A. D’Angelo et al.

S0 ∂H = grad div H − grad He ke ∂t and its validity is granted over any finite region V0 whose inside entropy is S0 . Now, by using the vectorial identity rot rot A = grad div A − ∇2 A, we can put the preceding relation into its final form S0 ∂H − ∇2 H = rot rot H − grad He ke ∂t

(7)

where the left side is more properly the heat equation for the vectorial quantity H, so that we can interpret the right side as a generalized diffusing force Fe acting on any individual component of the swarm Fe = rot rot H − grad He

(8)

Entering much more details we can see that the heat density H can be taken as same as the magnetic vector potential so that we can find a scalar quantity U ˆ holds and U can be interpreted such that the following relation rot H = −kU as the magnitude of the heat circulation inside the substratum. Since an autonomous robot is governed through its velocity, the mechanical control Fe of its operations can be understood as a positive frictional force.

3 Reaction Realising Control The robot change of direction which mechanically results from the energy and momentum exchange during contact, is actually driven by the force Fe = gVG , so that eq. (8) instantiates to ˆ gVG = −grad He − rot (kU)

(9)

and the actual velocity VG 8 of the robot results in summing up two components, the former referred to the velocity V on the original trajectory and the latter Vs due to contact reaction. Remember that, generally, the velocity of the robot always stems from two different components, the latter being generated by a vector whose direction is normal to the plane where the colony is operating. Hence, the above equation becomes ˆ ∧ grad U gVG = −grad He + k

(10)

where the shortened notations U = −rotz H and He = U + V0 p have been used. Thus, the dissipative function F of the fluid dynamic metaphor can be also interpreted as enthalpy; we have already seen that the internal energy U provides the heat circulation of the thermodynamic substratum so that the 8

We have chosen this notation to avoid misleading interpretations of formulas.

Motion Control of Dense Robot Colony Using Thermodynamics

93

velocity of any individual component is triggered by the free energy of the colony. Remember that its total internal energy is conserved and it results in supplying the substratum with a suitable enthalpy distribution and heat density. As a concluding remark one can always understand the colony collective task by introducing an electromagnetic metaphor where the heat flux is interpreted as magnetic vector potential whereas the enthalpy He is just the electrical scalar potential providing the robot colony with the goals to be collectively satisfied.

3.1 Adiabatic Process The following discussion is about the mechanism by which an individual robot can control the system properties by triggering its own individual behaviour. Specifically, we shall assume that the energy variation of an individual robot is due only to the energy exchange with other robots of the colony. Hence, the energy is conserved if we consider the swarm as a whole system.

(a) Aggressive

(b) Coward

Fig. 3 Braitenberg vehicles

In this case, the process involving all the individual robots of the swarm can be interpreted as an adiabatic process from the thermodynamic point of view. Because the heat change of the system dQ = T dS is null, we can conclude that the entropy of the system takes a constant value S0 , so that eq. (3) becomes −S0 dT = dG − V0 dp where V0 is the fixed volume occupied by the swarm. In this case, the free energy G takes the form G = pV0 − S0 T = W − S0 T using the definition of the quantity W discussed in sect. 2.1. Remembering the definition of the free energy G in terms of the heat flux H, the above equation can be resolved with respect to W , yielding to W = div H + S0 T

(11)

94

A. D’Angelo et al.

The equality chain appearing hereafter make use of some results presented in the preceding section. Avoiding cumbersome details, we have d W d t =

V · grad W = V · (grad div H + S0 grad T)   ˆ + ∇2 H − S0 ∂H = V · rot(−kU) k ∂ t e   ˆ ∧ grad U − g(V + Vs ) = V· k

ˆ ∧ grad U = grad W, because no heat flows outwards the volume But k boundary where the swarm is displaced, so that we can write 2

dW = −gV 2 − gV Vs dt

(12)

The right side of the above equation is built with two parts; the former is always negative whereas the sign of the latter depends on the sign of Vs . If we consider that the system exhibits a diffusing behavior if dW is positive and merging if negative, the result shows that the diffusion can be controlled by manipulating the sign of Vs . In the assumption of gas particle, used in the discussion, the force direction always belongs to the direction determined by the contact. With respect to the behaviour of Braitenberg vehicle two (fig. 3) [3, 11], the direction of Vs becomes the same with the reaction of the coward vehicle. On the other hand, the aggressive vehicle behaviour is opposite. Thus, if we switch these two behaviours accordingly to the circumstances, we can control the diffusion and merging of the whole system. It is natural that the aggressive and coward behaviour are fundamental to generate social behaviors, and the above result shows that the system properties can be manipulated by simple reaction to contact.

3.2 Application The main advantage of using group properties like density is that the control implementation results in a simple control unit. This characteristic is effectively used when it is applied to system that are composed with a large number of robots, possibly, thousands of them. Actually, such a system is under development but, for example, the smart dust project[12] proposes devices, whose size is a micro order scale, equipped with sensors and communication units and, moreover, the cost of one unit is quit cheap. If we focuse our attention on the microrobotics research, we can see mobile units that have been developed with many small devices. Thus, it is not an abstract assumption to realise a system with thousands of dust-like mobility-featured devices by shifting the discussion on how to control and use such systems. Here, we want to emphasize the quality of our method for this issue. If the number of robots are really huge, it is impossible to control the movement

Motion Control of Dense Robot Colony Using Thermodynamics

95

of each robot, but only a few parameters can be manipulated by relating a sensor to the appropriate parameters. In our case, the sign of Vs in eq. (12), i. e. the reaction to contact, is dominant for the whole behaviour of the system, so the system can be manipulated only by controlling this reaction. As a concrete application, we can suggest a kind of cloths that is composed of thousands of small robots. This cloth has a mobility due to the interaction with neighbor robots and it can be controlled through the above mentioned schema. Because this cloth can transform itself by the mobility of robots, the cloth can assist the movement of the wearing person as power assists suits.

4 Conclusions In this paper, the relationship between the diffusion and merging processes of a colony of robots and the behavior of individual robots has been investigated within a thermodynamic framework. Individual robots are supposed to get environmental information only by contact sensing so that they can be interpreted as gas particles. We have also sketched a method of robot control, reaction releasing, by assuming an adiabatic process which yields to the diffusion triggering by switching between aggressive and coward behaviors of Braitenberg vehicles. We have also discussed how the close connection between robot sensor and motor subsystems can be understood in terms of substratum within the abstraction of roboticle modeling. We have also considered the impact of the physical metaphores involved in the definition of the substratum.

References 1. Arkin, R.: Behavior-Based Robotics. MIT Press, Cambridge (1998) 2. Asama, H., Yano, M., Tsuchiya, K., Ito, K., Yuasa, H., Ota, J., Ishiguro, A., Kondo, T.: System principle on emergence of mobiligence and its engineering realization. In: IROS 2003, Las Vegas (NV), pp. 1715–1720 (2003) 3. Braitenberg, V.: Experiments in synthetic psychology. MIT Press, Cambridge (1984) 4. Brooks, R.: A robust layered control system for a mobile robot. IEEE Jour. of Robotics and Automation 2(1), 14–23 (1986) 5. D’Angelo, A., Ota, J., Pagello, E.: How intelligent behavior can emerge from a group of roboticles moving around. In: IROS 2003, Las Vegas (NV), pp. 1733–1738 (2003) 6. D’Angelo, A., Pagello, E.: Making collective behaviours to work through implicit communication. In: Casals, A., Dillmann, R., Giralt, G. (eds.) ICRA 2005, Barcelona (E), pp. 81–87 (2005) 7. D’Angelo, A., Pagello, E., Yuasa, H.: Issues on autonomous agents from a roboticle perspective. Journal of Intelligent and Robotic Systems (2008) 8. Latombe, J.C.: Robot Motion Planning. Springer, Heidelberg (1991)

96

A. D’Angelo et al.

9. LaValle, S.M.: Planning Algorithms. Cambridge University Press, Cambridge (2006) 10. Kinoshita, M., Watanabe, M., Kawakami, T., Yokoi, H., Kakazu, Y.: Macroscopic quantitative observation of multi-robot behavior. In: Computational Intelligence and Multimedia Applications, Yokosuka, Japan, pp. 190–194 (2001) 11. Pfeifer, R., Scheier, C.: Understanding Intelligence. MIT Press, Cambridge (1999) 12. Sailor, M.J., Link, J.R.: Smart dust: nanostructured devices in a grain of sand. Chemical Communications 11, 1375–1383 (2005) 13. Jantz, S.D., Doty, K.L., Bagnell, J.A., Zapata, I.R.: Kinetics of robotics: The development of universal metrics in robotic swarms. In: Florida Conference on Recent Advances in Robotics. Miami, USA (1997) 14. Kerr, W., Spears, D., Spears, W., Thayer, D.: Two formal gas models for multi-agent sweeping and obstacle avoidance. In: Hinchey, M.G., Rash, J.L., Truszkowski, W.F., Rouff, C.A. (eds.) FAABS 2004. LNCS, vol. 3228, pp. 111– 130. Springer, Heidelberg (2005)

Modeling of Self-organized Competition Hierarchy with Body Weight Development in Larval Cricket, Gryllus bimaculatus Shiro Yano, Yusuke Ikemoto, Hitoshi Aonuma, Takashi Nagao, and Hajime Asama

Abstract. Larval crickets develop a big body and high aggressiveness under isolated conditions, but develop a small body and low aggression under crowded conditions. Furthermore, their ensemble variance increases. It is known that the heavier cricket tends to win. Therefore, the two growth processes affect competition. We constructed a mathematical model for intraspecific competition based on the Bonabeau model, showing body weight effects on the winning rate. Results show that the population mean of body weight decreases under the condition of specific memory enhancement that places more emphasis on losing than winning. Furthermore, hierarchical effects in foraging efficiency increase the ensemble variance.

1 Introduction It is essentially necessary for multirobot system to maintain community structure so as to perform the cooperative function. Recently some studies intend to transfer multirobot through bottlenecks with recalibrating the bilateral relations between community structure , and it is difficult to reconstruct the prior community structure after transit through the bottlenecks[1][2]. Shiro Yano Graduate School of Engineering, The University of Tokyo e-mail: [email protected] Yusuke Ikemoto and Hajime Asama Research into Artifacts, Center for Engineering (RACE), The University of Tokyo e-mail: {ikemoto,asama}@race.u-tokyo.ac.jp Hitoshi Aonuma Research into Electronic Science (RIES), Hokkaido University e-mail: [email protected] Takashi Nagao Human Infomation System Laboratories, Kanazawa Institute of Technology e-mail: [email protected]

98

S. Yano et al.

Reconstructing the prior community structure is important in many circumstances, for example, guard patrol and waitstaff : robots always deploy themselves close to security area or table whereas they go and return between narrow hallway, bottlenecks, frequently. Now we focus attention to animals’ starategy of territory. Most of animals guard and memorize own territory so once they are away from home, they’ll be back again[3]. We specifically examine the competition of cricket G. bimaculatus. G. bimaculatus also have territory behavior : they guard territory for foraging behavior and courtship behavior. It has never examined that cricket will be back to same place again or not when cricket is away from home. However, they remember the dominance between crickets in few hours and can reconstruct dominance hierarchy after an absence of few hours[4]. So territory behavior of cricket is thought as a premitive form of the way that multirobot reconstructs task distribution not spatial distribution. It is known that agonistic behavior, such as aggressive behavior, constructs territory[3][5]. Cricket causes aggressive behavior to construct territory too, so we focus in the way how crickets fight to the finish. Regarding the growth process of cricket G. bimaculatus, the adult insects’ body size, activity, and aggression are strongly associated with alteration of their growth environment [8][9][10]. Moreover, they display distinctive and destructive intraspecific interaction and fighting behavior. Recent reports describe that crickets develop a large body and high aggressiveness under isolated conditions[8][11]. Conversely, they develop a small body and low aggressiveness under crowded conditions. Therefore, intraspecific competition seems to be an embedded ability that maintains variance among them. In this paper, we present a mathematical model of intraspecific competition based on Bonabeau model[13], and clarify how competition affects individual body weight growth, ensemble variance, and the population mean of body weight. In order to understand the way territory is constructed, we would like to know about if crickets treat win or lose in the same manner or not. In this regard, we deduce which is the long-continued memory about win or lose. At last, we conclude that crickets’ experience of loses is long-continued state.

2 Modeling of Body Size Development A logistic function describes evolution of animal body weight growth of some types[6]. Two studies have examined body weight increases in relation to effects of population density in the cricket G. bimaculatus[8][9] and an evolutionary logistic equation fitting them(see fig. 1). For this study, we use a logistic growth function written in the differential form as shown below: dmi = α (T, Pi )ri mi , (1) dt

Fig. 1 Discrete body weight increase points are measured[8]

$QF[aYGKIJVa=I?

Self-organized Competition Hierarchy with Body Weight Development

99

       



 ri = r0 1 −











2QUVaJCVEJaFCVGa=FC[? mi M β (T, Pi )





(2)

where M = const. [g] > 0 is the mature weight, mi (t) [g]> 0 is the current weight, r0 = const. [s−1 ] > 0 is the egg growth rate immediately after production, ri (t) [s−1 ]> 0 is the growth rate, T [◦ C] is the temperature, Pi [g/s] is the foraging efficiency, and α and β respectively signify a weighting function on the growth rate and mature weight (see Fig. 2). Experience shows that α and β are important factors for development[12]. Equation (2) includes M; however, an animal can not know its own mature weight at an interim period during growth. We redefine the weight growth equation without M as the following. dri r0 dmi =− dt M β dt   ri r0 =− α ri mi = −r0 α ri 1 − Mβ r0

(3)

Regarding eq. (3), we must devote attention to the fact that the factor β was removed. The reason is the mode of formula development: we take the condition “α and β as time-invariant if T and Pi are constant.” Consequently, d β /dt = 0. Such a condition is necessary for a logistic growth curve and construed as solitary growth or at a least low-crowd-density growth condition, effects of population density can be negligible. In contrast, it is compensated as eq. (4) when effects of the population density can not be negligible.   dri d (M β ) /dt = (ri − r0 ) α ri − (4) dt Mβ In addition, d (M β ) /dt is expanded as follows

∂ Mβ dM β = dt ∂T ∂ Mβ = ∂ Pi

dT ∂ M β dPi + dt ∂ Pi dt dPi dt

The temperature is constant for the experiments[8][9].

(5)

$QF[9GKIJV=I?

100

S. Yano et al.

Convergence point (critical weight) Mβ(T,P)

Slope of the tangent (growth rate) r0α(T,P)

2QUVJCEVJFC[=FC[? Fig. 2 Each curve shows growth hysteresis under five types of percentage of the protein rate in food. There are two other temperature data: 23 [◦ C] and 31 [◦ C]. All data are from an earlier study[9]

According to eq. (5), the stabilization points of ri are ri = r0 and ri = The stable convergence point of mi is   1 d (M β ) /dt . mifix = M β 1 − r0 α M β

d(Mβ )/dt α Mβ .

(6)

Consequently, when we calculate the body weights under effects of population density, we must know the data of body weight and growth rate under low-density M β and r0 α , and the time variation of the foraging efficiency dPi /dt.

3 Modeling of Intraspecific Competition As computers became powerful and sophisticated, numerous studies of many-body systems using computer simulation increased rapidly during the 1990s. Bonabeau studied how the hierarchy emerges in the course of agonistic behavior in an intraspecific community using computer simulation[13]. A few studies derived from his model were later performed[14][15]. Lacasa studied the most primitive part of Bonabeau model using a discrete mean-field approximation approach[14].In this section, we construct a model of time variation of the foraging efficiency dPi /dt using the parameter of hierarchy from the modified competition model.

3.1 Modified Bonabeau Model The most primitive equation of the Bonabeau model is described as follows[14]. 1 1 + exp(η (h j (t) − hi (t))) hi (t + 1) = (1 − ρ )(1 − µ )hi (t) Pi j (t) =

(7)

Self-organized Competition Hierarchy with Body Weight Development

+

ρ (1 − µ ) N −1

101

N

∑

  Pi j (t) (hi (t) + 1) + (1 − Pi j (t)) (hi (t) − F) , (8)

j=1; j=i

ρ = N/S, N: Number of animals, S: Field size [cm2 ]

In those equations, Pi j is the probability that attacker i will defeat individual j, time dependent variable hi (t) is the hierarchy (Lacasa calls it status), η > 0 is an adjustable strength of force parameter, ρ is the population density, (1 − µ ) is a relaxation factor of status, which is interpreted as the fading memory of individuals. In this discussion, we treat hi as the parameter reflecting a dominance hierarchy in the way that Bonabeau treated it in his study[13]. Equation (8) reflects that, after each combat, the winner’s hi increases by 1 and the loser’s hi decreases by F. In their models, hierarchy emergently becomes differentiated between individuals at the point of critical density ρc with phase transition (see Fig. 3). That is to say, hi yields instability under ρ > ρc . Therefore, minute fluctuations in the hierarchy trigger a phase transition. Minute fluctuations result from, for example, individual differences of agonistic behavior generated over time or innate individual differences of the body structure. Large individuals are known to tend to defeat small ones. We add this body size effect with hierarchical effect(see eq.(8) and fig.(8)) Q+ i j (t) :=

1  .   1 1 + exp η2 2Pi j (t) − mmij

(9)

m0 = mi (0) = 0.7 × 10−3(1 + 0.5ξ ), where

(10)

0 < ξ < 1 is a fluctuation factor.

In the equations above, m0 is the initial weight after egg hatching. Furthermore, m0 is affected by the mother’s condition, so adoption of individual differences is completely natural.

3.2 Foraging Efficiency Affected by Hierarchy Definition of social hierarchy includes that the dominant hierarchy is superior to that of the subordinate in foraging efficiency[7]. We define this tendency to the factor of foraging efficiency Pi in eq. (11): 

JKGTCTEJ[

   

̂

Fig. 3 Hierarchy organized by phase transition. (N=4, F=1)

̂

̂ ̂







FGPUKV[=EO?



102

S. Yano et al.

∂ Pi dhi δ t dPi = , (11) dt ∂ hi δ t dt where δ t is the interval of time required for agonistic behavior; also, δ t/dt is a frequency of fighting in a day. In eq. (11), we construct ∂ Pi /∂ hi as Fig. 4, where dhi /δ t is an outcome of eq. (8) under the condition “δ t = 1”.

3.3 Parameter Determination In this section, we determine parameter r0 α , M β , pi j (mi , m j ) and δ t/dt from experiments. 3.3.1

Determination of r0 α

From eq. (1),(2), we can approximately derive r0 α as follows. dmi = α (T, Pi )ri mi ≈ α (T, Pi )r0 (1 − 0)mi dt m0 mi ≈ ≪ 1, m0 ≈ 0.7[mg]) (for t ≈ 0, Mβ Mβ mi = m0 exp (r0 α t) = m0 10(r0 α t log(e))

(12)

Therein, r0 α is the slope of the tangent (see fig. 2). We referred the value of that parameter from [9]. 3.3.2

Determination of M β

The analytical solution of eq.(3) is ri =

r0 , 1 + exp(r0 α (t − τ0 ))

(13)

and eq.(2)

Fig. 4 How the foraging efficiency is affected by hierarchy, with the assumption that min(hi ), requires double the time to achieve foraging equal to solitary ones

HQTCIKPIa GHHKEKGPE[a2K V

     

̂





JKGTCTEJ[aJK V



Self-organized Competition Hierarchy with Body Weight Development 0.4 0.3 r0 α 0.2 0.1 0

103

0.4 0.3

r0 α 0.2 0.1

45 35 25 15 Protein Rate[%] 23 24 25 26 27 28 29 30 31 5 Temperature [ºC]

0 30 23

24

25

26

27

28

Temperature [ºC]

29

30

31

5

15

40

50

Protein Rate [%]

Fig.  6 Fitted by r0 α (T, P) = a + (P−c )2 (T −c )2 , a = 0.055, b = b exp − 2d 2T − 2d 2P T P 0.35, cT = 32.2, cP = 46, dT = 5.5, dP = 26

Fig. 5 Data referred from [9]

Fig. 7 Duration of growth period fitted to 2τ0 = 1 τ= (T −T 0) , Q0 =

120 100 80

Last molt day60 [day] 40

Q0 Q10 10 0.4, T0 =18.5, Q10= (P−u)2 s + t exp − 2v2 , s

20

= 2.3,t = 2.5, u = 42, v = 15. [9]

23 24 45 50 25 26 35 40 27 28 25 30 29 30 15 20 31 5 10

Temperature [°C]

Protein Rate [%]

M β = 1 + exp(r0 ατ0 ) .

(14)

Here, τ0 is the half period from hatch to adult; we referred the value of this parameter from [9]. 3.3.3

Determination of pi j

Nagao performed an experiment related to the effect of body weight on the winning rate in G. bimaculatus, as presented in Fig. 8. 3.3.4

Determination of δ t/dt

Fig. 8 Point distribution shows that the winning rate pi j is monotone increasing function of body weight ratio Wr . It is fitted by 1 pi j = 1+exp(3.3(1−W )) r

9KPPKPIaRGTEGPVCIG=㧑?

Herein, δ t/dt represents the frequency of agonistic behavior per day. Aonuma conducted an experiment related to the recovery speed crawl back from a lost battle in G. bimaculatus, as shown in Fig. 9. We use δ t/dt = 1 and µ = 4.47 ∗ 10−3 by this result.      













4CVKQQHaGPEQWPVGTỦDQF[YGKIJVU

S. Yano et al.

$GJCXKQTCN4GURQPUGU=?

104

6KOG=OKP? Fig. 9 Histogram showing that the avoidance rate requires 12 h to recover to 50[%]

4 Simulation and Analytic Prediction First, we run a computer simulation of the model in Matlab and compare the results to experimental data. Second, we discuss the model analytically.

4.1 Computer Simulation We determine other undefined parameters as η = 1, F = 2, N = 4, andS = 16. Simulation 1 shows a variance increase of the body weight on the last molt day. Simulation 2 demonstrates decreased body weight in the final weight. 4.1.1

Simulation 1: Comparison of Ensemble Variance

This simulation is designed to compare the ensemble variance of the final weight under solitary growth and crowded growth. As discussed previously, we set N = 4. The ensemble variance of solitary growth has roots in the fluctuation factor on m0 . 4.1.2

Simulation 2: Comparison of Population Means

This simulation compares the population means of the final weight under solitary growth and crowded growth. Bonabeau shows that the population mean of hierarchy becomes zero under (F = 1). We show (F = 0.5) and (F = 2) specifically. The mean weights under crowded growth is greater than under solitary growth in (F = 0.5), this is opposite to (F = 2).

Self-organized Competition Hierarchy with Body Weight Development













 a

9GKIJV=I?

9GKIJV=I?





  







105









a

   







9GKIJV=I?

9GKIJV=I?



 





=I? 







a





 

2TQVGKP=㧑?

















a





 



2TQVGKP=㧑?









6GORGTCVWTG=͠?

Fig. 11 Self organization of body weight, which has a root from hierarchical foraging efficiency(N=4, F=2)





















  ̂ ̂ ̂ ̂ 





6GORGTCVWTG=͠?







9GKIJV=I?

9GKIJV=I?



=I? 

Fig. 10 Body weight of solitary individuals, which has root from initial fluctuation of body weight



 



   

̂



̂  



2TQVGKP=㧑?



 











6GORGTCVWTG=͠?

Fig. 12 Subtract the mean weight of group growth individuals from the mean weight of solitary ones under 0 < F < 1







2TQVGKP=㧑?



 











6GORGTCVWTG=͠?

Fig. 13 Subtract the mean weight of group growth individuals from the mean weight of solitary ones under F > 1

4.2 Analytic Approach: A Qualitative Feature from eq. (6) β) Herein, M β is the body weight of solitary growth of individuals, so δ i = − r01α d(M dt are correction terms of the density effect(eq. (6)). In addition, (max(δi ) − min(δi )) reflects the variance of individuals’ body weights affected by the density effect. Furthermore, N1 ∑Ni=1 δi reflects the mean value of the individuals’ body weights affected by the density effect. We discuss these two equations and make a qualitative prediction below.

4.2.1 Approach 1: Magnitude Correlation of max(δi ) and min(δi )     max(mi )-min(mi ) crowded − max(mi )-min(mi ) solitary shows that the ensemble   variance of body weights. It is equivalent to max(δi )-min(δi ) . The difference on β δi arises from factor dM dt . d (M β ) d (M β ) dP dhi dP = and > 0, dt dP dh dt dhi

(15)

106

S. Yano et al.

dMβ β dhi so that difference on dM dt originates from the factors dP and dt . Merkel redMβ ported that dP shows positive and negative value[9]. To see qualitative trend on the ensemble variance, it is sufficient to show that any two terms are different (s.t. dh j dhi dt = dt | i = j). Moreover, it is readily apparent that the bifurcation of hi satisfies dh

j i ( dh dt = dt | i = j). Consequently, the variance of individuals’ body weights increases or decreases.

4.2.2

Approach 2: Discriminant Positive and Negative of (− N1 ∑Ni=1 δi )

The discriminant positive and negative of (- N1 ∑Ni=1 δi ) are equivalent to those of N dh ( N1 ∑Ni=1 dh dt ). The foregoing study demonstrated that (∑i=1 dt < 0) when (F > 1)[14]. Therefore, the mean value of an individual’s body weight decreases when losing a game has a greater effect than that of winning a game (F > 1).

5 Discussion and Conclusion For the analyses described in this paper, we constructed a mathematical model of intraspecific competition based on the Bonabeau model[13] to clarify how competition affects individual body weight growth, ensemble variance, and population mean of body weight. We added the effect of body weight on the winning rate to the simplest Bonabeau model and the effect of the foraging rate, as influenced by the dominance hierarchy. Results demonstrated that fluctuation of initial body weight causes bifurcation of hierarchies. Using the analytical approach with eq. (6), we showed that hierarchical bifurcation increases the variance of body weight and that F > 1 causes a mean value decrease of body weight. Actually, F > 1 means that losing a game has a greater effect than winning a game, which is not improbable. This study includes a quantitative comparison because biological data for Gryllus bimaculatus are lacking. For example, we substitute τ and r0 α from group individuals for those values from solitary ones. Actually, r0 α is probably fit in solitary ones because competition might be negligible in the neighborhood of t ≈ 0. However τ might not. We expect to perform further laboratory experiments. Several studies have examined density effects on body size[16].We expect to perform more simulations on other animals as well. Initial individual differences (e.g., physical structure, informational imbalance, and so on) engender further diversity with time in general terms, as shown by developmental phenomena such as those occurring through evolutionary progress.

References 1. Kurabayashi, D., Osagawa, K.: J. Robotics Society of jap 23, 376–382 (2005) 2. Ishiguro, A.: Robotics and Autonomous Systems 54, 641–650 (2006) 3. Cain, M., et al.: 2nd Rev. edn. W W Norton & Co Ltd. (2002)

Self-organized Competition Hierarchy with Body Weight Development 4. 5. 6. 7. 8. 9. 10. 11. 12. 13.

107

Adamo, S.A., et al.: J. Exp. Biol. 198, 1691–1700 (1995) Rutte, C., et al.: Trends in Ecology & Evolution 21, 16–21 (2006) Grossman, M., Bohren, B.B.: J. Heredity 76, 459–462 (1985) Wilson, E.O.: Sociobiology. Harvard University Press (1982) Iba, M., Nagao, T., Urano, A.: Zool. soc. jap. 12, 695–702 (1995) Merkel, G.: Oecol. 30, 129–140 (1977) Simmons, L.W.: J. Anim. Ecol. 56, 1015–1027 (1987) Nagao, T.: Proc. 69th Anim. Meeting of the Zool. Soc. Jap. 15, 89 (1998) Davidowtiz, G., D’Amico, L.J., Nihout, H.F.: Evol. & Devel. 5(2), 188–197 (2003) Bonabeau, E., Theraulaz, G., Deneubourg, J.L.: Bulletin of Mathe. Biol. 58, 661–717 (1996) 14. Lacasa, L., Luque, B.: Physc. A. 366, 472–484 (2006) 15. Ben-Naim, E., Vazquez, F., Redner, S.: European Physc. Jour. B. 49, 531 (2006) 16. Dash, C.M., Hota, K.A.: Ecol. 61, 1025–1028 (1980)

Intelligent Mobility Playing the Role of Impulse Absorption Jae Heon Chung, Byung-Ju Yi, and Chang Soo Han J.H. Chung et al.

*

Abstract. In reality, biomechanical systems such as the human-body, mammal, and birds physically collide or interact with environment. Intelligent mobility inherent in biomechanical systems plays the role of absorbing the impulse that happens due to collision with environment. The impulse transmitted to the lowerextremity of biomechanical systems is first distributed to multiple contact points of foot. Subsequently, jointed leg structure absorbs the impulse as the form of momentum. Thus, the proximal joints far from the contact points receive less amount of impulse. This fact has been understood in an intuitive manner without any analytical ground. This work proposes a Newton-Euler based closed-form impulse model for biomechanical systems in case of multiple contacts. This model is applied to analyze the characteristic of the intelligent mobility of biomechanical system through an example of a landing Ostrich’s leg. Furthermore, potential application of the impulse model for multi-chained linkage systems is addressed.

1 Introduction Impact due to collision between the environment and the biomechanical system occurs at the contact point during the transition from motion to constrained motion. The interacting force at the contact point is difficult to control, and the modeling and control of impact has been considered as an important issue in the field of robotics. When human and animal contacts environment, actually there are many simultaneous contact points. When a bird lands on the ground, two or three toes of the bird’s foot experience simultaneous ground contact. Also when the human grasps an object or two humans shake their hands, more than two fingers collide with an object or fingers of others, simultaneously. In general contact problem like the above mentioned cases, multiple ground contacts occur more frequently as compared to the single ground contact. Fig. 1 (a) and Fig. 1(b) show the poses of the human skeleton when he is walking and running. Fig. 1(c) and Fig. 1(d) show the Ostrich’s body and its foot. Legs of the Human and Ostrich consist of large- and small-sized bones. Those articulated Jae Heon Chung and Byung-Ju Yi School of Electrical Engineering and Computer Science, Hanyang University, 1271 Sa-1 Dong, Sang-rok Gu, Ansan, Korea e-mail: [email protected] Chang Soo Han Department of Mechanical Engineering, Hanyang University

110

J.H. Chung et al.

nature of proximal bones plays an important role to minimize or absorb impact in collision with the environment. At multiple contact points, toes absorb impulse by decentralizing the impulse, and subsequently the jointed leg structure absorbs the impulse as the form of momentum. Thus, the trunk (or main-body) far from the contact points receives less amount of impulse. However, so far this fact has been understood in an intuitive manner without any analytical ground.

(a) Human walking (b) Human running (c) Ostrich’s skeleton (d) Ostrich’s foot

Fig. 1 Human and Ostrich zo yo

zo

xo

yo xo

Floating

Floating

Floating

Virtual

Z

Z Y

Y X

X

(a) Structure

(b) Kinematic model in the floating body

Fig. 2 Multi-chained biomechanical system

Figure 2 shows the kinematic model for the leg structure of the human or the ostrich. This system is floating on the air and comes to contact the ground at multiple points. The kinematic model of this system can be depicted as Fig. 2(b), which includes virtual coordinates representing the floating behavior. Human and animal have several toes. In this figure, branch structures are described as toes. In order to analyze the nature of intelligent mobility absorbing the impulse, two subjects need to be investigated as follows. (1) Modeling of external and internal impulse model for multiple contacts (2) Analysis of the leg structure contributing to impulse absorption To date, methods to evaluate the external impulse have been proposed by several researchers [1-4]. External impulse for multiple contact cases has been also reported [5]. However, several researchers have ignored the coupled external

Intelligent Mobility Playing the Role of Impulse Absorption

111

impulse among multiple contact points. When the robotic mechanism collides with environment, not only the end point, the joints of the system also experience impulsive forces or moments. The internal impulse models at the joints by using Newton-Euler equations have been proposed [6-8]. However, only the internal impulse model for single contact case has been reported. There is no prior work on the internal impulse model for multiple contact cases. In this work, we propose a Newton-Euler based closed-form impulse model for biomechanical system in case of multiple contacts and analyze intelligent mobility playing the role of impulse absorption. Section 2 reviews the dynamic modeling of multi-chained biomechanical systems with multiple contacts. External impulse model is reviewed in section 3. Section 4 introduces a closed-form internal impulse model for multiple contact cases. In section 5, the internal impulse model is applied to analyze the merits of the kinematic structure of biomechanical system such as a landing Ostrich. Furthermore, in section 6 potential application of the impulse model for multi-chained linkage systems is addressed with examples for contact tasks of humanoid robots and obstacle avoidance scheme of multiple mobile robots. Finally, conclusion is drawn in section 7.

2 Dynamics In this section, we derive the dynamic model of robotic systems with a floating body. The dynamic model of the whole robot system with multiple contacts can be expressed as * ⎤⎦ φ& − ∑ ⎡⎣ i GφvI ⎤⎦ T = ⎡⎣ Iφφ* ⎤⎦ φ&& + φ& ⎡⎣ Pφφφ T

N

T i

FI

(1)

i =1

where φ denotes the Lagrangian coordinates of the system and N is the number of contact points. i F I denotes the impulsive external force at the ith contact point.

⎡⎣ Iφφ* ⎤⎦ and ⎡⎣ i Iφφ* ⎤⎦ denote the inertia matrix of the whole system and the ith chain, * * ⎤⎦ and ⎡⎣ i Pφφφ ⎤⎦ denote the inertia power array of the whole sysrespectively. ⎡⎣ Pφφφ

tem and the ith chain, respectively. The Lagrangian coordinates consists of the virtual joint vector ( φ v ) and the motor-controlled joint vector ( φ a ).

3 External Impulse Model for Multiple Contacts Most generally, the impact is partially elastic in the range of 0 < e < 1 . When the coefficient of restitution ( e ) is known, the relative velocity of the colliding bodies immediately after impact can be obtained. The component of the increment of the relative velocity along a vector, which is normal to the contact surface, is given by T

( Δv1 − Δv2 )

T

ni = − (1 + e )( v1 − v2 ) ni

(2)

where v1 and v2 are the absolute velocities of the colliding bodies immediately before impact, and Δv1 and Δv2 are the velocity increments immediately after impact.

112

J.H. Chung et al.

When a robot system interacts with environment at multiple points, (1) can be rewritten as the following partitioned form:

⎡ &T * & ⎤ T I va ⎤ ⎡φ&&v ⎤ ⎢φ ⎣⎡ Pvvv ⎦⎤ φ ⎥ N ⎢ ⎥+ T − ∑ ⎡ ⎡⎣ viGφvI ⎤⎦ ⎡⎣ aiGφvI ⎤⎦ ⎤ i F I , ⎥ ⎣ ⎦ && ⎢ ⎥ * I aa ⎦ ⎢φ ⎥ φ& ⎡ P ⎤ φ& i =1 ⎣ a ⎦ ⎢⎣ ⎣ aaa ⎦ ⎥⎦

⎡T ⎤ ⎡ I T = ⎢ v ⎥ = ⎢ vv ⎣T a ⎦ ⎣ I av

(3)

where T v is a null vector because the virtual joints are not activated, and T a denotes the joint torque vector at the motor-controlled joints. So, (3) can be decomposed into two components; one for the virtual joints and another for the active joints By eliminating the acceleration vector ( φ&&v ) of the virtual joint, the dynamic model in terms of the motor-controlled joint set can be expressed as T T * * ⎤⎦ φ&& + φ& ⎡⎣ Paaa ⎤⎦ φ& − ∑ ⎡⎣ iGavI ⎤⎦ i F I T a = ⎡⎣ I aa a a a N

(4)

i =1

where

T T T * * * * ⎡⎣ I aa ⎤⎦ = I aa − I av I vv−1 I va , φ& ⎡⎣ Paaa ⎤⎦ φ& = − I av I vv−1φ& ⎡⎣ Pvvv ⎤⎦ φ& + φ& ⎡⎣ Paaa ⎤⎦ φ& , a a

∑ ⎡⎣ G N

i

vI a

i =1

⎤⎦ = I av I vv−1 ∑ ⎡⎣ viGφvI ⎤⎦ − ∑ ⎡⎣ aiGφvI ⎤⎦ . N

T

T

i =1

N

T

i =1

The relationship between the velocity of the ith contact point and the motorcontrolled joint velocity can be expressed as ⎡ i vI ⎤ & (5) i vI = ⎣ Ga ⎦ φa

where ⎡⎣ iGavI ⎤⎦ denotes the Jacobian relating the ith contact point’s velocity to the motor-controlled joint velocities. From (4) and (5), the velocity increment at the ith contact point is obtained by the following kinematic relationship. i

* ⎤⎦ Δ v I = ⎡⎣ iGavI ⎤⎦ Δφ = ⎡⎣ iGavI ⎤⎦ ⎡⎣ I aa

∑ ⎡⎣ G

−1 N

i

i =1

vI a

⎤⎦

T i

Fˆ I .

(6)

Assume that the robot impacts on a fixed solid surface, then the absolute velocity and the velocity increment of the fixed surface are always zero ( v2 = Δv2 = 0 ). Substituting (6) and (2) and congregating them from i=1 to i=N, we have ⎡ 1 Fˆ I ⎤ ⎢ ⎥ ⎢ 2 Fˆ I ⎥ ⎢ ⎥ ⎢ M ⎥ ⎢ Fˆ ⎥ ⎣N I ⎦

T

⎡ J11T ⎢ T ⎢ J12 ⎢ M ⎢ T ⎢⎣ J1N

T J 21 T 22

J M J 2TN

⎡ 1v J NT 1 ⎤ ⎡ n1 ⎤ ⎢ ⎥ T ⎥⎢ L J N 2 ⎥ ⎢ n2 ⎥ ⎢ = − (1 + e ) ⎢ ⎥ O M ⎢ M ⎥ ⎢ ⎥⎢ ⎥ T ⎢ L J NN ⎥⎦ ⎣⎢ n N ⎦⎥ ⎣ L

* ⎤⎦ ⎡⎣ j GavI ⎤⎦ . where J ij = ⎡⎣ iGavI ⎤⎦ ⎣⎡ I aa −1

T

T I 2

v

T I

O

⎤⎡n ⎤ ⎥⎢ 1 ⎥ ⎥ ⎢ n2 ⎥ ⎥ ⎢ M ⎥ (7) ⎥⎢ ⎥ T ⎥ n ⎢ ⎥ Nv I ⎦⎣ N ⎦

Intelligent Mobility Playing the Role of Impulse Absorption

113

When the friction on the contacting surface is negligible, the impulse always acts at the ith contact point along the normal vector ni . Thus, we have i

Fˆ I = i FˆIn n i

(8)

From (7), we derive the magnitude of the impulse along the normal vector at the ith contact point as follows i

where n = ⎡⎣ n1

T

T

n2

FˆIn = −

(1 + ei ) i vTI ni

T ni ⎡⎣ J1Ti

T L n N ⎤⎦ .

T ⎤⎦ n J 2Ti L J Ni

(9)

T

4 Internal Impulses Model for Multiple Contacts When a robotic system collides with the environment, the joints of the system experience internal impulses along constrained directions of joints. The magnitude of the internal impulse cannot be measured unless we attach sensors at the contacting surface of joints. Fig. 3 Link with N br branches

j th

 j  1  j  2

j

j

th

 j  Nbr

Link j

rj 1

Cj

th

Fˆ j 1

th

Fˆ j rj

Fˆ j 2

Fˆ j  Nbr

Consider a serial chain with Nbr branches as shown in Fig. 3. The internal impulse model for such multiple contact case is introduced. Using the internal impulse model, we can analyze the internal force transmitted to links and momentum change of the system. By integrating the Newton-Euler equation with respect to time, the relationships between the velocity increments and the impulses in the jth coordinate frame attached to the jth link are obtained as

m j Δ j vC j = − j Fˆ j + ∑ ⎡⎣ j + brj R ⎤⎦ N br

j + br

Fˆ j + br

(10)

br =1

⎡ C j I ⎤ Δ jω j = − jτˆ j + ∑ ⎡ j + brj R ⎤ j + brτˆ j + br − j rj × j Fˆ j + j rj +1 × ∑ ⎡ j + brj R ⎤ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ br =1 br =1 Nbr

Nbr

j + br

Fˆ j + br ,

(11)

C where m j and ⎡ j I ⎤ are the mass and the moment of inertia of the jth link, respec⎣ ⎦ tively. The velocity of the jth mass center and the angular velocity of jth link are

114

J.H. Chung et al.

denoted as j vC j and jω j , respectively. ⎡⎣ j + brj R ⎤⎦ is the rotation matrix transforming the (j+br)th coordinate system to the jth coordinate system. The increments of the velocity of the jth mass center and the angular velocity of the jth link are obtained by T Δ j vC j = ⎡⎣ 0j R ⎤⎦ ⎡⎣ j GφC ⎤⎦ Δφ&

(12)

T Δ jω j = ⎡⎣ 0j R ⎤⎦ ⎡⎣ j Gφω ⎤⎦ Δφ& ,

(13)

where ⎡⎣ j GφC ⎤⎦ and ⎡⎣ j Gφω ⎤⎦ denote the Jacobians relating the velocity of the mass center j vC j and the angular velocity jω j of the jth link to the joint velocity vector, respectively. Substituting (12) and (13) into the left-hand sides of (10) and (11) yields m j Δ j vC j = m j ⎡⎣ 0j R ⎤⎦ ⎡⎣ j GφC ⎤⎦ ⎡⎣ Iφφ* ⎤⎦ T

∑ ⎣⎡ Gφ

−1 N

i =1

* ⎡ C j I ⎤ Δ jω j = ⎡ C j I ⎤ ⎡ 0j R ⎤ ⎡ j Gφω ⎤ ⎡ Iφφ ⎤ ⎦⎣ ⎦ ⎣ ⎦ ⎣ ⎦⎣ ⎦ ⎣ T

vI

i

⎤ ⎦

T i

Fˆ I

∑ ⎣⎡ Gφ

−1 N

vI

i

i =1

If the kth link experiences the external impulse at the and (11) are modified as

Nk

⎤ ⎦

(14) T i

Fˆ I .

contact points, then (10)

mk k ΔvCk = − Fˆk + ∑ ⎡⎣ k + brk R ⎤⎦ k + br Fˆk + br + ∑ ⎡⎣ 0k R ⎤⎦ q FˆI Nbr

Nk

br =1

q =1

(15)

⎡ Ck I ⎤ Δ kωk = − kτˆk + ∑ ⎡ k + brk R ⎤ k + brτˆk + br − k rk × k Fˆk ⎣ ⎦ ⎣ ⎦

(16)

Nbr

+ rk +1 × ∑ ⎣⎡ k + brk R ⎦⎤ k + br Fˆk + br + ∑ k rq × ⎣⎡ 0k R ⎦⎤ q FˆI , br =1

Nbr

Nk

br =1

q =1

(17)

k

(14) through (17) can be expressed as an augmented form as follows ⎡ Fˆ ⎤ ⎥ + [ B ] Fˆ I , ˆ τ ⎣⎢ ⎦⎥

[ D ] Fˆ I = [ A] ⎢ where the matrix ⎡⎣ A⎤⎦ is given by

⎡[ A ] [ 0] ⎤ [ A] = ⎢ AFF A ⎥ ⎣⎢[ TF ] [ TT ]⎦⎥

(18)

(19)

is identical to the case of a single contact [8]. However, the block matrices [ D ] and [ B ] with N k contact points in the kth link are different from that of the single contact case. It is formed as follows

Intelligent Mobility Playing the Role of Impulse Absorption ⎡ ⎡ v J11 ⎢⎢ ⎢ ⎢ v J 21 ⎢⎢ M ⎢⎢ ⎢ ⎣⎢ v J N 1 ⎡⎣ D ⎤⎦ = ⎢ ⎢ ⎡ w J11 ⎢⎢ J ⎢ ⎢ w 21 ⎢⎢ M ⎢⎢ ⎣⎢ ⎢⎣ w J N 1

v v

v

L

J 22 M

L

J1 N ⎤ ⎤ ⎥⎥ v J2N ⎥ ⎥ M ⎥⎥ ⎥⎥ ⎥⎥ v J NN ⎦ ⎥ J w 1N ⎤ ⎥ ⎥⎥ w J2N ⎥ ⎥ M ⎥⎥ ⎥⎥ w J NN ⎥ ⎦ ⎦⎥ v

O

JN2 L

w w

w

J12

J12

L

J 22 M

O

L

JN2 L

, and

0 ⎡ ⎢ k L ⎢ ⎡⎣ 0 R ⎤⎦ ⎢ 0 ⎢ M ⎡⎣ B ⎤⎦ = ⎢ ⎢ ⎢ k r1 × ⎡ 0k R ⎤ L ⎣ ⎦ ⎢ ⎢ 0 ⎢ M ⎣

where v J ji = m j ⎣⎡ 0j R ⎤⎦ ⎡⎣ j GφC ⎤⎦ ⎡⎣ Iφφ* ⎤⎦ ⎡⎣ i GφvI ⎤⎦ and −1

T

115 ⎤ ⎥ ⎡ R⎤ ⎥ k ⎣ ⎦ ⎥ 0 ⎥ ⎥ M ⎥ k k rN k × ⎡⎣ 0 R ⎤⎦ ⎥ N + k ⎥ ⎥ 0 ⎥ M ⎦ 0

k 0

,

T

C J ji = ⎡ j I ⎤ ⎡⎣ 0j R ⎤⎦ ⎡⎣ j GφωI ⎤⎦ ⎡⎣ Iφφ* ⎤⎦ ⎡⎣ i GφvI ⎤⎦ . ⎣ ⎦ Therefore, the closed form internal impulse model at the joints of the multichained manipulator with multiple contact points are formulated as −1

T

T

w

⎡ Fˆ ⎤ −1 ⎢ ⎥ = [ A] [ D ] − [ B ] Fˆ I . ˆ ⎣⎢ τ ⎦⎥

(

)

(20)

As noted in (20), the internal impulse at joints is directly affected by the external impulse ( Fˆ I ) and also function of robot geometry.

5 Simulations Figure 4(a) shows the kinematic model of Ostrich’s leg in Fig. 1(c). Table 1 shows the kinematic and dynamic parameters for its structure. It is assumed that the endpoints of the two distal links ( l3 and l5 ) collide with the ground by the velocity of 1.0 m/s. The coefficient of restitution e is assumed to be 0.8. Table 1 Kinematic and dynamic parameters 2

Link

Length[m]

Mass[kg]

Inertia[kg·m ]

l1

0.7 0.7 0.3 0.15 0.15

5 5 2 1 1

0.20424 0.2042 0.0150 0.0019 0.0019

l2 l3 l4 l5

The distal part of the Ostrich’s leg consists of two foot. The front foot is a rigid bone and the rear foot consists of two bones connected by one joint as shown in Fig. 4(a). We consider three different configurations of the feet as denoted in Fig. 4(b) through Fig. 4(d). Table 2 shows the magnitude of the external and internal impulses for the four simulation cases of the Ostrich’s leg. 1 FˆI and 2 FˆI are

116

J.H. Chung et al.

(a) Kinematic model

(b) Case 1

(c) Case 2

(d) Case 3

Fig. 4 Kinematic model and different configuration of the ostrich leg

external impulses at the front and the rear foot, respectively. Fˆ1 , Fˆ2 , and Fˆ3 denote the internal impulses at the proximal joint, the ankle joint, and the articulated joint of rear foot, respectively. In general, the amount of external impulse ( 2 FˆI ) exerted on the articulated (rear) foot ( l5 ) is smaller than that ( 1 FˆI ) of the front foot ( l3 ). Thus, we can conclude that more articulated links receive less external impulse. The amount of external impulse for each foot is almost identical for the three cases. On the other hand, the case 2 has the smallest internal impulse at the proximal joint as compared to the other cases. This is because the internal impulse transmitted from the two distal links ( l3 and l4 ) to the link l2 is cancelled partially due to the opposite sign. However, this case is not desirable since the intersecting position of link 3 and link 4 might be damaged due to the fighting of the two internal impulses. Table 2 External and internal impulses

Impulse

External [ N ⋅ s ]

Internal [ N ⋅ s ]

ˆ 1 FI

ˆ 2 FI

Fˆ3

Fˆ2

Fˆ1

Case 1

1.5272

0.7115

0.2442

0.4818

0.1732

Case 2

1.6103

0.6916

0.5019

0.2582

0.1396

Case 3

1.4650

0.8232

0.3429

0.5565

0.1679

Case 4

1.2986

0.5878

0.2074

0.4227

0.1814

The foot configuration of the Ostrich leg is similar to the configuration of case 3. Upon impact on the end point of the real foot, the distal link ( l5 ) of the real foot plays the role of absorbing momentum. Thus, resultantly, the internal impulse

Intelligent Mobility Playing the Role of Impulse Absorption

117

transmitted to the upper link ( l4 ) is fairly reduced without damaging the branching position of the foot. Fig. 5(a) through Fig. 5(c) show the external and internal impulses for the cases 1 through 3 of Fig. 4(b) through Fig. 4(d). As denoted by arrows at joints, the amount of the internal impulse decreases as the link is far from the contact position. Furthermore, we take into account the effect of joint configuration of the given kinematic model. The configuration of the Ostrich’s leg is bended like the leg model of Fig. 5(d) that shows the external and internal impulses for the case 4. When comparing the straight-standing configuration (case 3) with the Ostrich’s leg configuration (case 4), the configuration of case 4 receives less external and internal impulses as shown in Table 2. This is because the links of the bended configuration tends to move more when an external impulse is given to the end of the feet. And thus it is able to absorb more impulse due to relatively much change of momentum. 0.4 0.2

1

FˆI

1

0.2

FˆI

0

0 -0.2

-0.2

Fˆ1

-0.4

Fˆ1

-0.4 -0.6 2

FˆI

-0.6 2

-0.8

FˆI

-0.8 -1

-0.8

-0.6

-0.4

-0.2

0

0.2 Fˆ3

0.4

0.6

0.8

Fˆ2

Fˆ3

-1

Fˆ2

-1.2

-1

1

(a) Case 1: parallel feet

-0.8

-0.6

-0.4

1

0

1

FˆI

0

0.2

0.4

0.6

0.8

(b) Case 2: symmetric feet 0.8

0.2

-0.2

FˆI

0.6

-0.2

0.4

Fˆ1

0.2

-0.4 -0.6

2

2

FˆI

FˆI

Fˆ1

0

-0.8

-0.2

-1

-0.4

-1.2

Fˆ3

Fˆ2 -0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

Fˆ3

-0.6

0.8

1

(c) Case 3: asymmetric feet

Fˆ2

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

(d) Case 4: different configuration

Fig. 5 External and internal impulse of the Ostrich’s leg

6 Distributed Passive Autonomy and Its Application The phenomenon that more articulated links receive less external and internal impulses and joint configuration also affects the amount of internal impulse can be named as “distributed passive autonomy”. In other words, the inherent intelligent structure of the biomechanical system not only passively adapts to environments, but also ensures stability. This is opposite to distributed active autonomy that requires control action executed by the measured sensor information.

118

J.H. Chung et al.

Contact tasks are inevitable in the humanoid motion as well as the human-body motion. Such distributed passive autonomy can be found in the motion control of multiple humanoids working together. The distal end of the humanoid’s upperextremity (i.e., hands) consists of multiple fingers which play the role of distributing and absorbing the external impulse created upon collision. The remaining impulse is then transmitted to the forearm and the upper arm. The optimal configuration of these arms would be more or less folded-shape so that the arm can absorb the impulse as the form of momentum efficiently. This is similar to the shape of the human body when it is ready to contact other agents. The role of distributed passive autonomy can be shortly described as “damper with stability” because the impulse is absorbed through the multi-chained structure and the resulting motion of the system is stable by distributing the impulse to multiple chains. Thus, such merits can be implemented to group behaviors, coordinated control, or distributed planning of multiple robotics systems. Specifically, avoidance of collision is essential when multiple robots work together at certain environment. Even upon collision between robots, the amount of impulses transmitted to each robot should be also minimized. Usually a potential field is employed to obstacle avoidance of motion planning of such systems. A repulsive force is generated and applied to a mobile robot that is approaching to some obstacles. Instead of using such potential field approach, the concept of virtual multiple chains that connect a mobile robot to other mobile robots or obstacles is employed. When a mobile robot detects some obstacle within the allowed range of safety, a virtual chain that consists of two links and two joints is created for each obstacle. The mobile robot can be visualized as having multi-chains mounted on its platform. Then an impulse is created for each obstacle and its amount is proportional to the estimated magnitude of the obstacle because the amount of impulse is proportional to the system mass. When any obstacle approaches the mobile robot within the range of safety, the obstacle exerts an impulse on the end of each chain. The amount of impulse transmitted to the main body of the mobile robot is calcur 1 Δt r lated according to (20). Next, a velocity equivalent to ΔV = ∫0 F dt is added to M the original velocity vector of the mobile robot. Thus, the resulting velocity of the mobile robot is the vector summation of the original velocity of the mobile robot and the additional velocity components created by multiple virtual chains. In this scheme, the two-linked chain plays the role of impulse absorption, which is equivalent to the role of damper.

7 Conclusions The contribution of this work is suggestion of a closed-form external and internal impulse model of the multi-chained robotic systems for the multiple contact case. We show the effectiveness of the proposed model by comparing the single contact case with multiple contact case. A biomimetic model resembling the toe of Ostrich was also analyzed. The lessons obtained from this work are summarized as follow;

Intelligent Mobility Playing the Role of Impulse Absorption

119

(1) More articulated links receive less external and internal impulses. (2) Joint configuration also affects the amount of internal impulse. (3) The proposed impulse model is applicable to distributed autonomy of diverse multiple robotic systems. Acknowledgements. This work was partially supported by the Korea Science and Engineering Foundation (KOSEF) grant funded by the Korea government (MEST) (No. R01-2008-000-11742-0) and was partially supported by the Research fund of HYU (HYU2008-T).

References 1. Walker, I.D.: Impact configuration and measures for kinematically redundant and multiple armed robot systems. IEEE Trans. Robot. Autom. 10(5), 670–683 (1994) 2. Liao, H.-T., Leu, M.C.: Analysis of impact in robotic peg-in-hole assembly. Robotica 16(3), 347–356 (1998) 3. Kim, J.H., Chung, W.K., Youm, Y.I.: Normalized impact geometry and performance index for redundant manipulators. In: Proc. IEEE Int. Conf. Robot. Autom., pp. 1714– 1719 (2000) 4. Ismaeil, O.M., Ellis, R.E.: Effects of non-tip external forces and impulses on robot dynamics. In: Proc. IEEE Int. Conf. Syst. Man Cyber., pp. 479–485 (1992) 5. Gravagne, A.R., Walker, I.D.: Towards impulsive manipulation: a general algebraic collision model for spatial robots. In: Proc. IEEE Int. Conf. Robot. Autom., pp. 1707–1713 (2000) 6. Zheng, Y.-F., Henami, H.: Mathematical modeling of a robot collision with its environment. J. Robot. Syst. 2(3), 289–307 (1985) 7. Wittenburg, J.: Dynamics of Systems of Rigid Bodies. B.G. Teubner, Stuttgart (1977) 8. Lee, S.H., Yi, B.-J., Kim, S.H., Kwak, Y.K.: Modeling and analysis of internal impact for general classes of robotic mechanisms. In: Proc. IEEE/RSJ Int. Conf. Intel. Robot. Syst., pp. 1955–1962 (2000) 9. Freeman, A., Tesar, D.: Dynamic modeling of serial and parallel mechanism/robotic systems, Part I-Methodology, Part II-Applications. In: Proc. 20th ASME Biennial Mechan. Conf., Tredns, Devel. Mecha., Mach., Robot., pp. 7–27 (1988)

Part III

Ambient Intelligence

Cognitive Ontology: A Concept Structure for Dynamic Event Interpretation and Description from Visual Scene Yuki Wakuda, Kosuke Sekiyama, and Toshio Fukuda

Automatic detection and description of events, particularly human behavior, is one of the most challenging issues, since event interpretation is highly dependent on the target of attention, which is not uniquely specified. In order to tackle this problem, we propose the concept of a “Cognitive Ontology” as a framework for a system that can automatically decide the attention focus and describe the events. A cognitive ontology is structured with conceptual units which are Entities and Relations, and these units enable robot endogenous attention fixation and jumps based on a networked Cognitive Ontology. In addition, we introduce exogenous attention fixation based on how the observed targets differ from a predicted pattern. In this process, the corresponding target of attention is updated and assigned to an event description buffer that consists of two Entities and one Relation. In this paper, we have developed and experimented with this holistic event interpretation process taking into account endogenous attention, exogenous attention and determined event description.

1 Introduction The purpose of this research is to achieve visual observation and automatic event description by a computer with sensors (henceforth referred to as a robot). If such a robot with an advanced interface were available, broad technical applications in security or human friendly robots, such as automatic Kosuke Sekiyama and Toshio Fukuda Dept. of Micro-Nano System Engineering, Nagoya University Furo-cho, Chikusa-ku, Nagoya, Aichi 464-8673 Japan Yuki Wakuda Dept. of Interfaculty Initiative in Information Studies, The University of Tokyo Annex 3F, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8654 Japan e-mail: [email protected]

124

Y. Wakuda et al.

live surveillance systems, could be expected. The event description is seen as giving a certain interpretation to the observed phenomena by eliminating and contracting unnecessary information. Therefore, in order for a robot to describe a phenomenon, proper symbol grounding to an original signal modality is necessary. The difficulty in this process is the polysemy of event interpretation; that is, there can be multiple interpretations of a single scene depending on the scale of the symbol that is selected for event description. In this paper, we propose the “Cognitive Ontology” which provides the robot with the position and scale of the attention target for event description, and debate its practicality.

2 Event Description from Visual Scene 2.1 Previous Research and Issues Most of the research related to event description from a movie scene has been done in the research domain of “video surveillance”. Much research has been done in, for example, multimedia video surveillance, such as detecting the scene change in a movie or a drama and then performing suitable index attachment (movie indexing research) [1, 2], extracting the desired scenes and combining these in order to shorten the video length [3], and describing the behavior of a person or an event automatically based on the animation observed in a public space or sport scene (video annotation) [5, 6, 4] . Our research aim is to enable a robot that coexists in an environment with man, to recognize human actions and describe events automatically. Robotic vision and ordinary video surveillance research have some common and some different problems with respect to event description. The first common problem is the difficulty of description due to the possibility of polysemic interpretation of the scene. The observed phenomenon and action (henceforth referred to as an “event”) can be given a polysemic interpretation depending on which concepts (which we call an “Entity” in this research) the subject’s attention is focused. Therefore, many interpretations exist depending on the subjective targets of attention and the scale or positional relationship between them. This is a common problem for visual recognition by both a robot and offline surveillance. However, the research on video surveillance tackles the problem of how to choose and describe an event based on the viewer’s interests from the information that exists in the video. The problem of interpretative polysemy in the video surveillance research domain is often fixed and cleared beforehand about the subjects (for example, “people”, “a car”, etc.) of events, such as sporting activities, criminal activities, and others [5, 11, 10, 6]. In contrast to the research domains mentioned above, when it comes to introducing a robot into the environment and to recognizing daily events, the robot is required not only to recognize a car or man as the target which is fixed in size, but also to perform various interpretations while the scale changes

Cognitive Ontology

125

continuously throughout a sequential scene in real time. Generally, researchers have introduced the “attention model” of robots to subjective interpretation as a solution to the fundamental problem of polysemic scene interpretation Previous research on image recognition has mainly dealt with the problem of how to recognize the behavior of a person and the events observed. However, this problem was seldom tackled as dynamic event recognition or description with reference to “multiple attention scales”. In this research, we propose a “Cognitive Ontology” as the conceptual structure for dynamic event recognition by a robot, and a system architecture that mutually complements active determination of the attention-target and event interpretation. In addition, we have evaluated the usefulness of the proposed method in an event description experiment using robotic vision.

2.2 Transition and Fixation Attention Target for Successive Event Interpretation Here we consider an example of event recognition, which is to determine human behavior by a robot. In our architecture, successive event description is performed along with this observation process when, for example, “man → face → hand”, is observed in sequence. How does a robot determine the Entities “face” and “man” and their positions and boundaries? In the case of a human, a part of the entity structure is given a priori (for example, human babies have the ability to perform simple face pattern recognition), and another part of the structure has been built up with complex concepts throughout their life from learning in the real world and is useful to recognize and describe these objects. Although we aim to acquire such a conceptual structure from a real environment, we have defined the conceptual structure beforehand in this paper. To represent the conceptual structure, a graphical structure composed of nodes and links is considered suitable to fix and move the attention target of observation. Although this graphical structure is very similar to a “semantic network”, we have called the proposed structure a “Cognitive Ontology”, since that is the structure of subjective concepts that is used in the recognition process of robots. The proposed Cognitive Ontology is located as a kind of domain ontology which is used for dynamic event interpretation and description.

2.3 Cognitive Ontology 2.3.1 Elements Comprising the Cognitive Ontology We define two components as elements required for the Cognitive Ontology; one is the Entity (ENT), which is the major attention target to be used for the component of event description, and the other is the Relation (REL), which shows the relative relationship between Entities. The structure of the

126

Y. Wakuda et al.

Cognitive Ontology is as a network with a mass of Entities: ENT linked by REL to each other. The Cognitive Ontology is different to an ordinary semantic network in that the Cognitive Ontology has a “Geometrical relation”, “Temporal relation”, “Semantic relation” and “Inclusive relation” to perform continuous object observation. CognitiveOntology = {ENT, REL}

(1)

We now discuss the elements necessary for the composition of ENT and REL. Henceforth, “∗” as in ENT(∗) is used as a wildcard character, with ∗ and ∗n representing an arbitrary single or n symbols, respectively. Although the Entity class and Relation class of the Cognitive Ontology can be considered similar to the concepts of “Firstness”, “Secondness” and “Thirdness” in the Upper Ontology model proposed by J.Sowa[17], the Cognitive Ontology with Entity: ENT and Relation: REL is different to Sowa’s model as highlighted by the following points. Firstness in Sowa’s model is defined as an instance with materiality and Secondness can exist as a qualifier instance for Firstness. However, in the Cognitive Ontology proposed in this research, the definition of an Entity is not necessarily given only to a material instance, and the modality (which is sometimes called “Qualia”) or information which is not a real thing in itself can be treated as an Entity, for example, information, an attribute, etc. The Relation Class of the Cognitive Ontology is unlike Sowa’s Thirdness, in that the Relation Class is not a strict and fixed knowledge representation structure and can be updated dynamically in the recognition process of a robot. In addition, the location is quite different in that it is aimed at the construction of a subjective conceptual structure for recognition, and not at the construction of “the common conceptual structure of the world” as in the target of a general Upper Ontological research. From the above, the definition of a Cognitive Ontology is different to that of a general Upper Ontology or fundamental Ontology. In what follows, we discuss the elements required for the composition of ENT and REL in a Cognitive Ontology framework. 2.3.2

Definition of Entity: ENT

We denote an element of Entity: ENT in the Cognitive Ontology with, hash (Hash key of this Entity), name (Name of Entity), center and ξc en t er (Center position coordinates of Entity and its uncertainty index), direction and ξd i r ec t i o n (Posture of Entity and its uncertainty index), scale and ξscale (Spatiotemporal scale of Entity and its uncertainty index), property (Observable information of Entity), port (Hash key of other Entities connected from this Entity). The Entity is expressed as the equation given below, while a definition of each parameter is given in Table 1. In addition, with respect to the composition elements, Entity and Relation, a vector value is denoted by a normal font character, while a bold character expresses a scalar value.

Cognitive Ontology

127

Table 1 Definition of Entity and its parameters Sign ENT.hash ENT.name ENT.center ENT.ξcenter ENT.posture ENT.ξposture ENT.scale

Definition Hash value of ENT information Name of ENT Center position of ENT Uncertainty of ENT.center Posture which its Entity can be observed by a robot Uncertainty value of ENT.posture Relative scale of Entity which is normalized by the size of robot. The dimension of scale is depend on the modality dimension of a robot ENT.ξscale Uncertainty value of ENT.scale ENT.property Observed pattern of ENT by a robot ENT.port Hash keys of other entities those are connected with this Entity

ENT = { hash, name, center, ξcenter , , posture, ξposture , scale, ξscale , property, port}

2.3.3

(2)

Definition of Relation: REL

Next, we define the “Relation” between Entities. An Entity has a relationship with other Entities, and this Relation is described as REL in the Cognitive Ontology. The REL consists of the following parameters, hash (hash value of Relation), name (name of Relation), property (class of Relation), direction and ξd ir ec tio n (direction of Relation and its uncertainty index), distance and ξdistance (spatiotemporal distance of Relation and its uncertainty index), and is described as follows. REL = { name, property, direction, ξdirection , distance, ξdistance } (3) A definition of the REL parameters is given in Table 2. 2.3.4

Interpretation of Class of Relation (REL.property)

Relation: REL shows the relative relation between Entities, and the relation can be interpreted from various viewpoints. For example, the relationship between “human” and its parts such as “head” or “foot”, etc., can be interpreted as geometrical relations. On the other hand, the parts are part of a “human”, and as such, the relation can be interpreted as an inclusive relation. Moreover, the arrangement of a part of “face”, such as “eyebrow”, “eye”, “mouth”, etc., will change with a time series transition, and these relationships can be interpreted as Temporal Transition Relations. There are many

128

Y. Wakuda et al.

Table 2 Definition of Relation between Entities and its parameters Sign REL.h ash REL.name REL.p r o p e r t y REL.d i r e c t i o n REL.ξdirection REL.s c a l e REL.ξscale

Definition Hash value of REL Name of REL Variety of REL Direction of REL Uncertainty index of REL.d i r e c t i o n Scale of between connected two ENT by this REL, for example, geometrical distance or time scale. Uncertainty index of REL.s c a l e

interpretations of Relation: REL depending on the subjectivity variation of the subject who gives the interpretation. It can be stated that the variation in interpretation is akin to the variations in human cogitation. In this paper, we take the robot’s ability to move and its sensors into consideration, and define the four major REL that are important for event recognition, namely RELg e o m e t r i c , RELs e m a n t i c , RELi n c l u s i v e and RELt r a n s i t i o n as following table.

Table 3 Definition of Relation properties of cognitive ontology Name Geometric Relation Transition Relation Semantic Relation Inclusive Relation

2.3.5

Sign RELgeometric RELtransition RELsemantic RELinclusive

Definition Geometric relationship between Entities Temporal transition between Entities Semantic relationship between Entities A relationship between multiple Entities and one Entity

An Example of Cognitive Ontology for Human Behavior Recognition

An example of a visualized Cognitive Ontology is shown in Fig. 1 as the Entity: “human”. Each Entity is given a geometric arrangement and uncertainty of the space arrangement, and the radius of the sphere reflects the scale of the uncertainty of the Entities to their position as shown in Fig.1. In addition, in the example we show that Entity : ENT has an Inclusive Relation: RELi n c l u s i v e with ENT(human) and parts of ENTface and also those parts of ENTface have Geometrical Relations: RELg e o m e t r i c with each other. In the following experiments, we examined dynamic event description using the Cognitive Ontology structure shown in Fig.1.

Cognitive Ontology

129

Fig. 1 An example of visualized Cognitive Ontology

2.4 The Entity Buffer for Event Description 2.4.1

Framework of Event Description

Therefore, attention target Entity is selected, and it will be updated with (0) the passage of time. We store the attention target Entity: ENTattn in (∗) (∗) ENT Cattn, and perform the selection of REL Cattn by the stored Entity. Then, the event is described by the set of ENT(∗) Cattn.name and REL(∗) Cattn.name.   (∗i) (∗i−1) Ev(0) = ENTCattn (+1), RELCattn (0) (4) However, i shows the number of conscious Entity buffers in the robot used for the Event description. We fixed the number i = 2 and all event description in this paper is performed with two Entities and one Relation. This corresponds to the grammatical construction of (S, V, O).   (2) (1) (1) Ev(0) = ENTCattn (+1), RELCattn (0), ENTCattn (+1)

(5)

Table 4 Definition of temporal Entity and Relation for recognition by a robot Class Sign Definition (∗i) Conscious Entity buffer ENTCattn (0) A Entity perceived by the system for temporal event recognition and description (∗i−1) Conscious Relation RELCattn (0) Selected relationship between temporal buffer Entities by the system for temporal event recognition and description

130

2.4.2

Y. Wakuda et al.

Expected Interaction and REL Update

In order to complete Event description, it is necessary to select the entity which (1) is related to current attention target: ENTCattn (0) and to complete the deter(1) mination process about RELCattn between them. Here, the current conscious attention target entity is ENTCattn and the entity with possibility of receiv(1) ing interaction with ENTCattn in the next frame of scene is called “Expected (2) interaction entity” (this is represented in ENTCattn as in formula 5). Here, the attention target in current state (Ev(0)) is ENTCattn (0). And the possibility of interaction in the next frame is evaluated based on the result of particle filter. According to particle filter-based movement analysis, the region with possibility of receiving interaction with ENTCattn (0) is caled “expected interaction region” and is represented as follows: if Expected interaction region (1)

(1)

= ENTCattn (+1).center ± ENTCattn (+1).scale (1 ≤ k ≤ N )

(6)

(∗)

N: Number of ENTUCattn (0).

and Entities ENT(l) are included in the “Expected interaction region” as follows: (l)

ENTUCattn (+1) ∈ Expected interacting area

(7)

here, l: Number of entities which is included in the expected interacting area.

then, Entities ENT(l) are “Expected interaction Entities”. Here, the Re(l) lation between ENTUCattn (+1) and ENTUCattn (+1) can be discussed for Evenet description. In this paper, two relations RELtransition and RELgeometric are used for evaluation of interaction. It is computable from the relation named Ev(0) and Ev(+1) as follows respectively. (1)

(1)

(8)

(2)

(1)

(9)

RELCattn transition = ENTCattn (+1) − ENTCattn (0) RELCattn geometric = ENTCattn (+1) − ENTCattn (+1)

From these process, REL representation can be fixed such as abstract repre{1,2} sentation “touch”, “with”, “move”. Moreover, when ENTCattn is specified, verb expression Entity such as “sit” “have” and “walk”can be selected for (1) RELCattn (+1). The proposed representation “Ev(0)” is updated continuously based on attention and the wholistic relationship of ENT and REL, and that process realize the dynamic Event description using Cognitive Ontology. The attention target and description of Ev(0) is updated through dynamic fixation (∗) and jump process of attention: ENTattn (0) between current frame(0) and predicted frame(+1) based on the connection of Entities in the Cognitive Ontology in real-time.

Cognitive Ontology

131

3 Experiment We conducted an experiment to evaluate the validity of the determination architecture for objects of attention based on the proposed Cognitive Ontology, and an event description system.

3.1 Experiment Term In the experiment, we examine a system developed to describe scenes of human behavior, such as “reading a book”, “drink coffee”, “scratch his head”, “turn over the page”. We created a Cognitive Ontology structure for this experiment. In this experiment, we focused on the following two evaluations. • The function of continuous update of the observation scale for event interpretation based on the Cognitive Ontology. • The function of continuous event description using selected entities: (∗) ENTUCattn .

3.2 Experimental Results According to the results of the experiment, the following three points were evaluated and the validity of an automatic determination for observation by the Cognitive Ontology and event description was obtained. 3.2.1

Searching Relevant Entity Based on Cognitive Ontology

At first, the system detects “face” using Haar-like pattern matching, then the system determines the position and scale of “hand”, “arm”, and “human”, relevant to “face” using the detected Entity: “face” as the basis of perception. 3.2.2

Estimation of REL by Determination of Two ENT

In the results shown in Fig. 2, at stage (a) in the event description state the following equation applies: EvCattn (0) (1)

(1)

(2)

=

{ENTCattn , RELCattn , ENTCattn }

=

{{ENTCattn

(right hand)

(coffee)

(have)

, RELCattn }, RELCattn }

(10)

At the next scene (b) in Fig. 2. The REL update is shown, that is REL: (1) “have” is updated to “drink” based on the determined ENTCattn and (2) ENTCattn .

132

Y. Wakuda et al.

Ev(0) (1) (2) = (ENTCattn , REL(1)Cattn , ENTCattn

)

= ("right hand", "have", "coffee" )

(a) Focus attention to moving object based on exogenious attention.

Ev(0) (1) (2) = (ENTCattn , REL(1)Cattn , ENTCattn

= ("face", "near", "coffee"

)

) 1 and 2

(b) REL.name: “drink” is selected as REL.temporal from ENT Cbuff .

Ev(0) (1) (2) = (ENTCattn , REL(1)Cattn , ENTCattn

)

= ("human", "drink", " coffee " )

(c) ENTtemporal was rescaled based on REL.inclusive of Cognitive Ontology, and focused on “human” and neglect other lower body components.

Fig. 2 Significant scene of event description using Cognitive Ontology

3.2.3

ENT Update by Determined REL

From the results (b) and (c) in Fig. 2, the description system selected “human” as an Entity related to RELCattn = “drink” in (c), and stopped the tracking of lower Entities of “human” ( for example, “face”, “hand”, etc. ). As the next process, ENT is updated with another proper Entity for event (1) description which is selected from Entities of ENTCattn (0) according to the (1) estimated description: RELCattn (0).name = “drink”.

4 Conclusion In this paper, we proposed a “Cognitive Ontology” which is a knowledge representation for a robot’s recognition of observed events. We also formulated the definition of the Cognitive Ontology and proposed an automatic event description framework using the Cognitive Ontology. In the experiment to

Cognitive Ontology

133

evaluate the proposed method and system, we show the validity of the active determination of the observation target using the Cognitive Ontology proposed in this paper. We also verified the problem of observation target determination in the experiment, where the robot’s attention is shifted between a pre-defined Entity in the Cognitive Ontology, and decided the scale of event description by the Cognitive Ontology. Furthermore, the event description method using a temporary Entity and Relation can determine the attention scale and attention target shift, and it was shown that a continuous event description is realized dynamically. In our future work, we aim to investigate an autonomous acquisition method of the Cognitive Ontology structure by the robot itself.

References 1. Zhai, Y., Shah, M.: Video scene segmentation using Markov chain Monte Carlo. IEEE Transactions on Multimedia 8(4), 686–697 (2006) 2. Radke, R.J., Andra, S., Al-Kofahi, O., Roysam, B.: Image change detection algorithms: a systematic survey. IEEE Transactions on Image Processing 14(3), 294–307 (2005) 3. Rav-Acha, A., Pritch, Y., Peleg, S.: Making a Long Video Short: Dynamic Video Synopsis. In: Proc. on Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference (CVPR), vol. 1, pp. 435–441 (2006) 4. Hongeng, S., Nevatia, R., Bremond, F.: Video-based event recognition: activity representation and probabilistic recognition methods. Computer Vision and Image Understanding 96(2), 129–162 (2004) 5. Cheng, F., Christmas, W.J., Kittler, J.: Detection and description of human running behaviour in sportsvideo multimedia database. In: Proceedings of 11th International Conference on Image Analysis and Processing, pp. 366–371 (2001) 6. Ngo, C.-W., Pong, T.-C., Zhang, H.-J.: On clustering and retrieval of video shots through temporal slices analysis. IEEE Transactions on Multimedia 4(4), 446–458 (2002) 7. Collins, R.T., et al.: A system for video surveillance and monitoring: VSAM final report, CMU-RI-TR-00-12, Technical Report, Carnegie Mellon University (2000) 8. Yu, X., Yang, S.X.: A study of motion recognition from video sequences. Computing and Visualization in Science archive 8(1) (2005) 9. Chen, G., Kotz, D.: A Survey of Context-Aware Mobile Computing Research. Dartmouth Computer Science Technical Report TR2000-381 (2000) 10. Dedeoglu, Y., Toreyin, B.U., Gudukbay, U., Cetin, A.E.: Real-Time Fire and Flame Detection in Video. In: Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 2, pp. 669–672 (2005) 11. Silveira Jacques Jr., J.C., Braun, A., Soldera, J., Raupp Musse, S., Rosito Jung, C.: Understanding people motion in video sequences using Voronoi diagrams: Detecting and classifying groups. Pattern Analysis & Applications archive 10(4), 321–332 (2007) 12. Cavanagh, P., Labianca, A.T., Thornton, I.M.: Attention-based visual routines: sprites. Cognition 80, 47–60 (2001)

134

Y. Wakuda et al.

13. Itti, L., Koch, C.: Computational modelling of visual attention. Nature Reviews. Neuroscience, 194–203 (2001) 14. Demiris, Y., Khadhouri, B.: Hierarchical attentive multiple models for execution and recognition of actions. Robotics and Autonomous Systems 54(5), 361–369 (2006) 15. Walther, D., Rutishauser, U., Koch, C., Perona, P.: Selective visual attention enables learning and recognition of multiple objects in cluttered scenes. Computer Vision and Image Understanding 100(1-2), 41–63 (2005) 16. John, P.: Biopsychology, 5th edn. Pearson, London (2005) 17. Sowa, J.: Distinction, combination, and constraints. In: Proc. of IJCAI 1995 Workshop on Basic Ontological Issues in Knowledge Sharing (1995) 18. Cutler, R., Davis, L.S.: Robust real-time periodic motion detection, analysis, and applications. IEEE Trans. on Pattern Analysis and Machine Intelligence 22(8), 781–796 (2000) 19. Mohan, A., Papageorgiou, C., Poggio, T.: Example-based object detection in images by components. IEEE Trans. on Pattern Recognition and Machine Intelligence 23(4), 349–361 (2001) 20. Ioffe, S., Forsyth, D.: Probabilistic methods for finding people. International Journal of Computer Vision 43(1), 45–68 (2001)

Subjective Timing Control in Synchronized Motion of Humans A Basic Study for Human-Robot Interaction Mitsuharu Nojima , Hiroaki Shimo, and Yoshihiro Miyake M. Nojima et al. 1

Abstract. The purpose of this paper is to clarify the internal phase correction mechanisms for timing control in a synchronization tapping task. Here, the synchronization error (SE) is directly controlled using our experimental method, and its response is measured as the temporal development of the inter tap onset interval (ITI). As a result, two types of internal phase correction mechanism are revealed. The first type shows a strong negative correlation between SE change and ITI change, suggesting a simple negative feedback mechanism. The second one does not show this correlation and disappears under the dual task condition. These results suggest that there are two types of timing control mechanism in the phase correction of tapping task.

1 Introduction Humans adapt flexibly to a dynamic environment. The timing control mechanism plays an important role in obtaining such an adaptation process, and the importance of timing control becomes clear if one considers coordinated human behavior, such as music, sports and communication. For example, Condon et al. showed that the mutual interaction between speech rhythm and body rhythm plays an important role in the communication between mother and infant [1]. Matarazzo [2], Webb [3] and Nagaoka [4] reported that the duration of utterance, the speed of utterance and the reaction time are synchronized among speakers. Furthermore, psychological estimation of the influence from timing of utterance, nodding and gesture is also studied in human-robot interaction (HRI) [5][6]. However, these timing mechanisms have been studied from an external viewpoint which is based on objective time. There are no reports about subjective timing from an internal viewpoint. Therefore, a new horizon of communication science and a new technology for realizing natural HRI are expected to be developed by the Mitsuharu Nojima, Hiroaki Shimo, and Yoshihiro Miyake Department of Computational Intelligence and Systems Science, Tokyo Institute of Tecnology, 4259 Nagatsuta, Midori, Yokohama 226-8502 Japan e-mail: [email protected]

136

M. Nojima et al.

research of this subjective timing mechanism. From these backgrounds, the purpose of this study is to clarify the subjective timing control as a basic study for HRI. In order to investigate this timing mechanism, a synchronization tapping task [7-9] in which the subjects are required to synchronize their taps with periodic stimuli as precisely as possible, has been widely used. The most interesting finding is the phenomenon that the onset of each tap precedes the onset of stimulus by several 10 msec [10-12]. This anticipatory phenomenon is called “Negative Asynchrony,” and has been replicated in a number of studies. The pressing-in-advance phenomenon, for which the subject is unaware, demonstrates that the motor command to the finger is generated before the onset of auditory stimulus, suggesting a processing of subjective timing control. Previous studies of this synchronization tapping task can be separated into two categories. The first category is the modeling of this timing mechanism [13-16]. The most important one was proposed by Mates [14][15], which is characterized as a subjectively controlled negative feedback model. The second category is its neural basis [17-19] and two types of timing process have already been clarified; a physically controlled process and a cognitively controlled process [20][21]. Here, we define the timing control requiring attentional resources as a cognitively controlled process, not requiring a physically controlled process. However, since Mates’ model is described by perceived subjective variables, such as internal phase error and internal period error, it is difficult to compare Mates’ model with its externally observed variables. Therefore, we developed a new experimental method which is able to control internal phase error directly and investigated the subjective timing mechanism based on the Mates’ model. Furthermore, in relation with our previous studies about timing process, we also investigated the influence of attentional resources to the timing mechanism, by applying the dual task method [22].

2 Experimental Task 2.1 Mates’ Model In this study, we are investigating the subjective timing control mechanism based on Mates’ model. The reason why we adopt Mates’ model is that timing mechanism of humans is explained by internally perceived subjective timing, and this model is systematically developed from previous timing control models. This Mates’ model can explain the timing mechanism in synchronization tapping as shown in Fig. 1. Upper part of the dash line and lower part of the dash line shows externally observable physical event and internally perceived subjective event, respectively. The subscript i stands for internal variables. We make all variables which describe Mates’ model into Table 1. The Mates’ model consists of two different mechanisms. One is the period error correction Eq. 1 and the other is the phase error correction Eq. 2. (1) Ti ( n ) = Ti (n − 1) − β ⋅ [Ti (n − 1) − {S i (n ) − S i ( n − 1)}]

Subjective Timing Control in Synchronized Motion of Humans

137

Ri ( n + 1) = Ri ( n ) + Ti ( n ) − α ⋅ ei ( n )

(2)

where Ti is the intrinsic period of the internal timekeeper. Si is the perceived internal timing of the external stimulus. Ri is the initiation timing of the internal motor command. ei indicates the internal synchronization error (internal phase error) which is defined as the time difference between Si and Ri. β and α are coefficients of feedback gain. ISI(k-1)

k

k+1

ISI(k)

ISI(k+1)

S

ITI(k)

ITI(k-1) -

R

ITI(k+1)

SE(k+1)

SE(k)

SE

External Internal Si

Ą Ć

ui

ui mi

mi

Ri

fi

fi

Fi

ei(k+1)

e i(k)

ei

Time

Fig. 1 Time scheme and definition of external (upper part) and internal (lower part) temporal variables of sensorimotor synchronization process and time diagram of the hypothesized mechanism of subjective synchronization [15] Table 1 Variables of sensorimotor synchronization model [15] Variable

Type

Description

External variables S Event

Occurrence of stimulus onset

ISI R

Interval Event

Inter stimulus-onset interval Occurrence of motor response onset

ITI SE

Interval Interval

Inter tap-onset interval Synchronization error

Internal variables ui Interval mi Interval Interval fi Si Event Event Ri Fi Event Interval ei ti

Interval

Transduction delay of stimulus Motor delay in execution of motor act Transduction delay of feedback information from already executed onset of motor act Temporal central availability of stimulus Initiation of motor command Temporal central availability of feedback Internal synchronization error (time difference between temporal central availability internal representations of some aspects of stimulus and response) Internal timekeeper (reference) interval

Here, we define ΔTi and Δei as follows. Δ Ti ( n ) = Ti ( n + 1) − Ti ( n )

(3)

Δei ( n ) = ei (n + 1) − ei ( n )

(4)

Then, Mates’ model can be transformed into Eq. 5. ΔTi ( n ) = − β ⋅ Δei (n ) − α ⋅ β ⋅ ei ( n )

(5)

This means that Mates’ model is a feedback control system. The change of the intrinsic period of the time keeper is modified based on the internal phase error and the change of it.

138

M. Nojima et al.

2.2 Transformation of Variables Since Mates’ model was described by internally perceived variables, it is difficult to directly verify the relevancy by measuring externally observed variables of the tapping task. Therefore, we should define the relationship between external variables and internal variables by transformed Mates’ model. First, we define the external variables that are used in synchronization tapping task. As shown in following, three variables, Inter Stimulus-onset Interval (ISI, Eq. 6, Inter Tap-onset Interval (ITI, Eq. 7) and Synchronization Error (SE, Eq. 8) are defined. Here S(n) indicates the n-th stimulus onset and R(n) indicates the n-th tap onset. (6) ISI (n ) = S (n + 1) − S (n ) ITI ( n ) = R ( n + 1) − R (n )

(7)

SE ( n ) = R( n ) − S ( n )

(8)

Here, we introduce two hypothesis. One is that we consider neurotransmission delay as constant. Mates treated transduction delay of stimulus and of feedback as random. However, the clear account or mathematical basis of them are not shown by Mates. Therefore, we these regard transmission delay as conatant. ΔTi ( n ) = Ti ( n + 1) − Ti ( n ) 　　　 = {Ri (n + 2) − Ri (n + 1) + α ⋅ ei (n + 1)}

　　　　　 − {Ri (n + 1) − Ri (n ) + α ⋅ ei (n )} 　　　 = ITI i (n + 1) + α ⋅ ei (n + 1) − ITIi (n ) − α ⋅ ei (n ) 　　　 = ΔITI i (n ) + α ⋅ Δei (n )

(9)

Here, in Eq. 9, considering the constancy of the transduction delay between motor command and tap motion is constant, the perceived internal ΔITIi(n) can be replaced with the external variable ΔITI(n). Therefore, from Eq. 9, we obtain the following. ΔTi (n ) = ΔITI ( n ) + α ⋅ Δei ( n )

(10)

Using this relationship between internal variable and external variable, Mates’ model Eq. 5 is transformed into Eq. 11. (11) ΔITI (n ) = −(α + β ) ⋅ Δei ( n ) − α ⋅ β ⋅ ei (n ) Furthermore, it is necessary to relate internal phase error ei(n) to externally observable variable. Thus, we introduce the other hypothesis as below. In this experimental task, subjects were instructed to control their internal synchronization error as low as possible. This means that SE is considered to be the time difference between externally observed synchronized state and internally perceived synchronized state. This is also supported by the negative asynchrony phenomenon. Thus, by regarding the averaged SE as the timing of the internally synchronized state, it becomes possible to consider the time difference between tap onset and the

Subjective Timing Control in Synchronized Motion of Humans

139

averaged SE as the internal phase error. Therefore, ei is able to be defined as following, by using external variables. (12) ei ( n ) = 0 ⇔ SE ( n ) = mean SE And, internally perceived phase error ei is able to be defined as following, by using external variables SE(n). ei ( n ) = SE (n ) − mean SE (13) By these two hypothesis, Mates’ model can be related to externally observable variable.

2.3 Design of the Experimental Task In this study, we define a modified synchronization tapping task which is composed of two different stages (Fig. 2). In the first stage (FISI condition), subjects conduct the normal synchronization tapping task under fixed ISI condition for realizing the internally synchronized state. The time difference between the internal synchronized state and the external synchronized state is defined by calculating the mean of SE. This mean SE is regarded as the timing that internal phase error ei is 0. In the second stage (FSE condition), we experimentally control the synchronization error SE to the averaged SE in order to fix the internal phase error to 0. Then SE is changed by ΔSE to fix the internal phase error ei at ΔSE. By this method, we can control the internally perceived synchronization error. In this experimental task, by analyzing the relationship between the controlled variable (ΔSE) and the response variable (ΔITI), the internal timing mechanism Eq. 11 is investigated to verify Mates’ model. The internal phase error is controlled at 0 in the m-th tap and is controlled at ΔSE after the (m+1)-th tap. ei ( m ) = 0, ei ( m + 1) = ΔSE

(14)

Here, the Eq. 11 is described as follows. ΔITI (m + 1) = −( α + β ) ⋅ ΔSE

(15)

After the (m+2)-th tap, using an integer k, with (k > m+1), ei ( k ) = ΔSE , ei ( k + 1) = ΔSE

(16)

the Eq. 11 becomes as follows. ΔITI (k ) = −α ⋅ β ⋅ ΔSE

(17)

Therefore, if the timing mechanism is realized based on Mates’ model, it is expected that ΔITI and ΔSE have a linear relationship as shown in Eqs. 15 and 17. Thus, we can analyze the relationship between Mates’ model and experimental data. It is possible to compare relationship between ΔITI and ΔSE obtained in this experiment, with Mates’ model directly. Therefore, it becomes possible to analyze timing mechanism quantitatively.

140

M. Nojima et al.

3 Methods 3.1 Procedure Subjects were instructed to tap in synchrony with the periodic auditory stimuli as precisely as possible. They pressed a button using their right index finger. One set consists of 30 taps. We used 25 taps for analysis, excluding the initial unstable 5 taps. These 25 taps are divided into the first 20 taps, the following 1 tap and the remaining 4 taps. In these initial 20 taps, subjects conducted the normal tapping task with the fixed ISI condition (FISI condition). Here, we measured SE and calculate SE1 defined as the time difference between the internal synchronized state and the external synchronized state, as shown in Eq. 18. SE1　= (mean SE in the FISI condition) (18) Then, in the following 1 tap, the internal phase error was fixed to 0 by controlling SE at SE1 (FSE-1 condition). For the remaining 4 taps, the internal phase error was fixed to ΔSE by controlling SE at SE2 (FSE-2 condition) as shown in Eq. 19. SE2 = SE1　+ ΔSE (19) In this experiment, 900msec ISI was used in the FISI condition. Thirteen different ΔSE (-90, -75, -60, -45, -30, -15, 0, 15, 30, 45, 60, 75 and 90msec) were used in the FSE-2 condition.

ITI[sec]

1

0

ǻSE

SE[sec]

0.1

ITI SE

-0.1 0.5

meanSE -0.2

FISI condition 0

Tap number

FSE condition m

-0.3

Fig. 2 Design of the experimental task

3.2 Data Analysis Due to fluctuations of observed variables, it is difficult to verify Eqs. 15 and 17 independently. Thus, we newly define ΔITIα as a response to the change of the internal phase error, and use for analysis. This ΔITIα is amount of ITI change per one Tap, calculated from the slope of regression line of ITI change in the fixed SE condition.

3.3 Dual Task Method In this study, we used the dual task method [22] to estimate the influence of attentional resources to the timing mechanism. We compared the single task condition

Subjective Timing Control in Synchronized Motion of Humans

141

(normal tapping) with the dual task condition (normal tapping with silent reading). In the single task condition, subjects conducted the synchronized tapping task with their eyes closed. In the dual condition, the subjects had to silently read text, that was scrolling in front of them, on a computer screen. After each trial, the subjects were tested concerning the contents of this text (Table 2). The Japanese language version of “The Mouse in the Mountain” written by Norbert Davis was used for this silent reading task. Furthermore, the test results were compared to that of only silently reading (Table 3). As a results, percentage of correct answers in reading task with tapping significantly decrease from that of without tapping(p0

SE[sec]

ITI[sec]

SE

-0.1 0.5

ǻITIĮ

-0.2

0

-0.3 0

5

10

15

20

25

Tap number ITI

ITI[sec]

0.1

ǻITIĮ

SE 1

0

ǻSE0 c: ΔSE desiredDistance, where desiredDistance is the distance between two well-positioned robots forming a triangular lattice. Otherwise the force is repulsive. Only nearby robots at a distancei j < 1.3 ∗ desiredDistance are considered neighbors. The resulting force is given by the sum of all the forces from the neighboring robots: Fi = ∑ Fi j

(2)

j

At each cycle, the application of this virtual force Fi modifies the virtual velocity: Fi (3) vi (t + Δ t) = vi (t) + Δ t m which is translated by the low-level controller to set the motor speeds of the robot.

3.2 Alignment State In this state the robot aligns with neighboring robots in order to get a common heading within the group. Robots reach the Alignment state when the following conditions have been fulfilled: • For every neighboring robot (robots with distancei j < 1.3 ∗ desiredDistance) desiredDistance − e < distancei j < desiredDistance + e

(4)

where e is the tolerance error in the positioning. Once a robot has reached the Alignment state, a larger value of e is used. • A given robot must have at least two neighbors in order to enter the Alignment state. The Alignment algorithm works as follows. Each robot calculates the average heading of its neighboring robots that are already in the Alignment or Moving states. Those in the Positioning state might be turning to position themselves, so their heading is not considered. A proportional controller using the difference between personal heading (headingi) and the average heading (averageHeading) is applied: vangulari = K3 ∗ (averageHeading − headingi) The linear velocity is set to 0.

(5)

208

I. Navarro et al.

vlineari = 0

(6)

All robots in the Alignment state apply the same controller, so by simply matching the heading locally a global common heading eventually arises at the group level. In principle robots might detect the distance and direction to their neighbors using an on-board relative positioning system as described in [15]. While it is more difficult to determine the heading of their neighbors, robots can couple relative positioning with communication in order to get this information [16]. Communication is also necessary to know the state of the neighboring robots. Once robots meet Positioning state and Alignment state conditions, they enter into the Moving state. The alignment condition is that the error in the heading with respect to the average heading of its neighbors must be under a certain level (e2 ): |headingi − averageHeading| < e2 .

3.3 Moving State In this state robots should share a common heading. Thus, by applying a virtual force in the forward direction, they will move together. In addition, drift within the formation could result in some positioning error with respect to the neighbors so the sum of the forces Fi j used in the Positioning state is also applied: Fi = K4 ∑ Fi j + K5 FForwardi

(7)

j

where K4 and K5 are two constants. As in the Positioning state, a local control law (see Section 3.5) translates this Fi force into motor speeds. In this manner, local rules applied by every robot result in a common, global movement of the complete formation. The group velocity is determined by K5 FForwardi .

3.4 Dispersion State Clustering phenomena occur when robots are stuck in the middle of the lattice and have insufficient force to push the other robots around them. This phenomenon is due to local minima of the inter-robot forces and can be solved by adding stochastic noise to the controllers, as in [1]. It can be minimized by setting the appropriate values in the control parameters, but it is not guaranteed that it will never happen. Experimentally it was observed that depending on the initial configuration of the robots, this phenomenon was more or less likely to take place, with higher chances for larger formations. It also appears when a robot is introduced into the middle of an already formed lattice, since the force that it creates on its neighboring robots is not sufficient to modify the created structure, and the robot finds itself in a deadlock. In order to overcome the clustering phenomenon, a solution was chosen which first identifies and then solves the problem. Each of the robots maintains a variable that represents an estimation of the probability of being in a cluster. When a robot

A Distributed Scalable Approach to Formation Control

209

identifies itself as being part of a cluster it enters into the Dispersion state, leaving the formation; this is accomplished by navigating to the border of the formation and not allowing the other robots to react to its position. As result, the cluster disappears and the robot rejoins the formation at the border, entering into the Positioning state. The probability of being part of a cluster is increased if the average distance to the closest neighbor is below a certain threshold and keep decreasing over successive time steps, otherwise this probability is decreased. Every step the robot verifies whether it is in a cluster by randomly choosing a value between 0 and 1 and comparing this value with the probability of being in a cluster. This randomized process represents a noisy perturbation on the controller as proposed in [1]. The process of detecting the clustering randomly reduces the probability of two nearby robots detecting the clustering at the same time and leaving the cluster simultaneously, which results in a hole in the formation.

3.5 Low-Level Controller A given virtual velocity, generated by both the Positioning and Moving algorithms, must be translated into wheel movements. The low-level controller is designed for mobile robots with differential drive configuration. The controller is inspired by a similar one described by Hsu [10], but augmented to allow backwards movement. The angle (θi ) and magnitude (|vi |) of the virtual velocity are the inputs for getting these two velocities:

vangulari

vlineari = K1 ∗ |vi | ∗ cos(θi ) ⎧ ⎪ ⎨K2 (θi + π ), if θi < −π /2 = K2 θi , if π /2 > θi > −π /2 ⎪ ⎩ K2 (θi − π ), if θi > π /2

(8)

(9)

The sum of the linear and angular velocities is translated to motor speeds taking into account the kinematics of differential drive robot as follows: smotor−righti = vlineari + B ∗ vangulari

(10)

smotor−le f ti = vlineari − B ∗ vangulari

(11)

where B is half the distance between the two wheels, smotor−righti is the speed of the right motor, and smotor−le f ti is the speed of the left motor. Any other controller based on the inverted kinematic model could be used instead of the proposed one.

4 Experiments In order to test the algorithm up to seven Khepera III robots were used. These robots have an on-board Linux system with wireless LAN communications, 2 motors in

210

I. Navarro et al.

differential drive configuration, a ring of 9 infrared sensors, and 5 ultrasound sensors. The size of these robots is 12 cm in diameter. Experiments were first done using the Webots simulator [13], with a realistic model of the robots. This allowed us to perform the experiments in a fast way, tuning the different parameters easily. Experiments in simulation were performed working with formations of 40 robots. The simulator has a built-in relative positioning system that gives information about the distance and direction to neighboring robots within line-of-sight. It also allows the exchange of information between them. This simulates an omnidirectional infrared relative positioning system, such as the one presented in [15], which allows for the simultaneous exchange of messages and assessment of range and bearing information. In our real robot experiments, in order to detect the positions of nearby robots and exchange information between them, an overhead camera tracking system was used to emulate on-board relative positioning. A camera was connected to the ceiling looking down at the arena and linked to a computer. The computer used a tracking tool which sent the information on the positions of the robots via UDP packages using the wireless LAN system. This system has an error of up to 1 cm in position and 1 degree in angle. Robots received these packages and calculated the relative positions to their neighbors taking into account occlusions and maximum communication range. In addition Gaussian noise was added to more accurately emulate a real on-board relative positioning system. The values for the noise were chosen according to the previous characterization of a relative positioning system in [15], using a standard deviation of 0.1 radians for the bearing and 10% of the distance for the range. This emulated system could be replaced with a recently developed system which offers comparable performance. The positioning information was updated approximately every 300 ms (a delay due to the image acquisition and processing). This value is quite high and according to [16] it might represent the bottleneck of the system, since robots cannot move very quickly without risk of oscillation. We ran 30 simulated experiments and 20 real robot experiments with an interrobot desired distance of 60 cm. The 20 real-world experiments were carried out using seven Khepera III robots that were placed in a small cluster. Another set of 20 experiments were done in Webots simulation with a model of the Khepera III robot and the same starting poses as the real-robot experiments. The last 10 experiments correspond to simulations with 40 robots with initial positions in a square lattice and random headings. An image of seven real Khepera III in formation can be seen in Fig. 3a, while in Fig. 3b, a larger simulated formation is shown.

5 Results The 50 experiments we ran worked quite well, since robots were eventually able to create a triangular lattice formation that was able to move as a whole in an emergent direction decided upon by all robots.

A Distributed Scalable Approach to Formation Control

(a)

211

(b)

Fig. 3 Two formations: (a) seven Khepera III robots, (b) forty Khepera III robots in simulation Table 1 Results on average for the different type of experiments Mean Mean Position Error Angle Error (m) (rad) 7 Real Robots 0.052 0.283 7 Simulated Robots 0.043 0.123 40 Simulated Robots 0.056 0.488

Group Time Time Time Speed Positioning Alignment Moving (m/s) 0.011 36.8 % 27.9 % 35.3 % 0.018 26.5 % 55.3 % 18.2 % 0.009 51.2 % 28.2 % 20.6 %

For each of the experiments, four metrics were analyzed: mean position error (averaged over the different robots), mean heading error (averaged over the different robots), speed of the center of mass of the formation (group speed) and the internal state of the robot (Positioning, Alignment or Moving). Dispersion state was not differentiated and considered as Positioning state. The averages over multiple runs of the first three values as well as the percentage of time in each internal state are given in Table 1 for the different experiments. They are determined by taking the values after some initial starting time, once the formation is already created. This time was set manually to 40 s in the case of experiments with seven robots, and 100 s for experiments with 40 robots. As can be seen, the average values do not change much between real robots and simulation in the case of 7 robots. Both mean position error, and group speed are quite similar. Group speed has a value of around 1 cm/s, which (taking into account that the maximum enforced speed for a robot is 3 cm/s) is a good result. The mean position error is about 5 cm, 8.3% of the desired distance of 60 cm, which is fairly impressive considering the dynamic nature of the system and the 10% range error from relative positioning. The percent of time that on average robots stay in each internal state shows differences between reality and simulation. We suspect this may be due to imperfections in the simulation model. The communication between real robots is asynchronous, with delays and lost packets, while communication in the simulator is synchronous and with no delays or losses. A better communication model would likely give closer results.

212

I. Navarro et al.

0.25 0.2. 0.15

1

0. 5

0.1 0.05 0 0

group speed [m/s]

mean heading error [rad]

mean position error [m]

1.5 0.3

10

20

30

40

time [s]

50

60

70

80

0.025 0.02 0.015 0.01 0.005

0

0

10

20

30

40

50

time [s]

60

70

80

0 0

10

20

30

40

50

60

70

80

time [s]

Fig. 4 Evolution of mean position error, mean heading error and group speed in an experiment using seven real robots

Experiments with 40 robots have similar results to those with just seven. Mean position error, internal state and (more importantly) group speed are quite similar, demonstrating empirically that the algorithm can scale well with increasing numbers of robots, thanks to their control laws that are distributed and local. The mean angle error is larger in the case of 40 robots, as well as the time in the Positioning state; the reason for this might be that, because of propagation of error in bearing measurements and the high convergence time, all the robots are not heading in the exact same direction. However this does not seem to be a major problem since the formation does not split. The workings of the algorithm can be understood better by observing the graphs of Fig. 4 that show the average values of the mean position error, mean heading error, and group speed over time for an experiment with seven real robots. In the initial phase of the experiment, the mean position error and mean heading error are quite big and most of the robots are in the Positioning state. After some time (around 15 seconds) the mean position error decreases significantly, but the mean heading error remains quite large. The explanation for this is that most robots are already well placed but still need to align themselves. After around 40 seconds, the mean heading error also decreases, indicating that the robots are aligned and beginning to move, as can be seen in the group speed graph. Once robots start moving the group speed and mean heading error have oscillations. This is due to heading errors gradually increasing while moving, requiring the formation to periodically stop and re-align. When it is properly aligned, it starts moving again. This pattern can be seen in all of the experiments done, both in simulation and with real robots. Some videos of real world and simulation experiments can be found at http://138.100.76.231/inaki/dars08/index.html

6 Conclusions and Future Work The algorithm for the control of robot formations presented here appears to work reasonably well according to our experimental results. A formation with an internal triangular lattice structure is created, with an external shape which is not predefined. Working with seven real robots, we observed that a formation emerges, just using

A Distributed Scalable Approach to Formation Control

213

local rules, local sensing and minimal communication, and that the formation is able to agree on a common direction of movement with no global information. Experiments in simulation with 40 robots show the scalability with respect to the number of robots in the formation. There is no evidence that the algorithm will not work for larger groups of robots although its performances might be gracefully decreasing with large swarms. Both group speed and mean error in position have reasonable values in simulation and real experiments. Scalability can be an interesting property in certain applications such as distributed search and mapping. In addition, the absence of any leader and the use of many robots make the formation tolerant to individual robot failures. The controllers could be improved using fuzzy controls to eliminate the discrete changes in behavior produced when jumping from one state to another in the FSM. Stability properties of the formation should be analyzed. Another possible improvement would be deciding the direction of movement not by all the members of the formation, but by those that have more relevant information. For instance, robots on the periphery of the formation could have more information about the environment and could lead the rest in the right direction, without any need for communication just by making the others copy their headings. In this way, obstacle avoidance at the group level could be implemented. Acknowledgements. I. Navarro is sponsored by Madrid Region with a Ph.D. grant (FPI2005), that allowed him to develop this work at EPFL. I. Navarro and F. Mat´ıa are partially sponsored by Spanish Ministry of Science (ROBONAUTA project DPI2007-66846-C02-01). J. Pugh has been sponsored by a Swiss NSF grant (contract Nr. PP002-116913).

References 1. Arkin, R.C.: Motor schema-based mobile robot navigation. The International Journal of Robotics Research 8, 92–112 (1989) 2. Balch, T., Arkin, R.: Behavior-based formation control for multi-robot teams. IEEE Transactions on Robotics and Automation 14(6), 926–939 (1998) 3. Balch, T., Hybinette, M.: Social potentials for scalable multi-robot formations. In: IEEE International Conference on Robotics and Automation, vol. 1, pp. 73–80 (2000) 4. Das, A., Fierro, R., Kumar, V., Ostrowski, J., Spletzer, J., Taylor, C.: A vision-based formation control framework. IEEE Transactions on Robotics and Automation 18(5), 813–825 (2002) 5. Fax, J., Murray, R.: Information flow and cooperative control of vehicle formations. IEEE Transactions on Automatic Control 49(9), 1465–1476 (2004) 6. Fredslund, J., Mataric, M.J.: A general algorithm for robot formations using local sensing and minimal communication. IEEE Transactions on Robotics and Automation, Special Issue on Multi Robot Systems 18(5), 837–846 (2002) 7. Fujibayashi, K., Murata, S., Sugawara, K., Yamamura, M.: Self-organizing formation algorithm for active elements. In: SRDS 2002: Proceedings of the 21st IEEE Symposium on Reliable Distributed Systems (SRDS 2002), pp. 416–421. IEEE Computer Society Press, Los Alamitos (2002)

214

I. Navarro et al.

8. Hayes, A.T., Martinoli, A., Goodman, R.M.: Distributed odor source localization. IEEE Sensors Journal 2(3), 260–271 (2002); special Issue on Artificial Olfaction, Nagle, H.T., Gardner, J.W., Persaud, K. (eds.) 9. Howard, A.: Multi-robot mapping using manifold representations. In: IEEE International Conference on Robotics and Automation, vol. 4, pp. 4198–4203 (2004) 10. Hsu, H., Liu, A.: Multiagent-based multi-team formation control for mobile robots. Journal of Intelligent and Robotic Systems 42, 337–360 (2005) 11. Kostelnik, P., Samulka, M., Janosik, M.: Scalable multi-robot formations using local sensing and communication. In: Proceedings of the Third International Workshop on Robot Motion and Control, 2002. RoMoCo 2002, p. 319–324 (2002) 12. Martinson, E., Payton, D.: Lattice formation in mobile autonomous sensor arrays. In: S¸ahin, E., Spears, W. (eds.) Swarm Robotics 2004. LNCS, vol. 3342, pp. 98–111. Springer, Heidelberg (2005) 13. Michel, O.: Webots: Professional mobile robot simulation. Journal of Advanced Robotics Systems 1(1), 39–42 (2004) 14. Naffin, D.J., Sukhatme, G.S.: Negotiated formations. In: International Conference on Intelligent Autonomous Systems, pp. 181–190 (2004) 15. Pugh, J., Martinoli, A.: Relative localization and communication module for small-scale multi-robot systems. In: IEEE International Conference on Robotics and Automation, USA, pp. 188–193 (2006) 16. Pugh, J., Martinoli, A.: Small-scale robot formation movement using a simple on-board relative positioning system. In: Khatib, O., Kumar, V., Rus, D. (eds.) International Symposium on Experimental Robotics, Rio de Janeiro, Brazil. Springer Tracts in Advanced Robotics, vol. 39, pp. 297–306 (2006) 17. Pugh, J., Martinoli, A.: Inspiring and modeling multi-robot search with particle swarm optimization. In: IEEE Swarm Intelligence Symposium, pp. 332–339 (2007) 18. Reynolds, C.W.: Flocks, herds, and schools: A distributed behavioral model. Computer Graphics 21(4), 25–34 (1987) 19. Sahin, E.: Swarm robotics: From sources of inspiration to domains of application. In: S¸ahin, E., Spears, W.M. (eds.) Swarm Robotics 2004. LNCS, vol. 3342, pp. 10–20. Springer, Heidelberg (2005) 20. Spears, W., Spears, D., Hamann, J., Heil, R.: Distributed, physics-based control of swarms of vehicles. Autonomous Robots 17, 137–162 (2004) 21. Spears, W., Spears, D., Heil, R., Kerr, W., Hettiarachchi, S.: An overview of physicomimetics. In: S¸ahin, E., Spears, W.M. (eds.) Swarm Robotics 2004. LNCS, vol. 3342, pp. 84–97. Springer, Heidelberg (2005) 22. Spears, W.M., Heil, R., Spears, D.F., Zarzhitsky, D.: Physicomimetics for mobile robot formations. In: AAMAS 2004, pp. 1528–1529. IEEE Computer Society, Los Alamitos (2004) 23. Tanner, H., Jadbabaie, A., Pappas, G.: Stable flocking of mobile agents, part i: fixed topology. In: Proceedings. 42nd IEEE Conference on Decision and Control, vol. 2, pp. 2010–2015 (2003)

Guiding a Robot Flock via Informed Robots Hande C ¸ elikkanat, Ali Emre Turgut, and Erol S ¸ ahin

Abstract. In this paper, we study how and to what extent a self-organized mobile robot flock can be guided to move in a desired direction by informing some of the individuals within the flock. Specifically, we extend a flocking behavior that was shown to maneuver a swarm of mobile robots as a cohesive group in free space avoiding obstacles in its path. In its original form, this behavior does not have a preferred direction and the flock would wander aimlessly in the environment. In this study, we extend the flocking behavior by “informing” some of the individuals about the desired direction that we wish the swarm to move. The informed robots do not signal that they are “informed” (a.k.a. unacknowledged leadership) and instead guide the rest of the swarm by their tendency to move in the desired direction. Through experimental results obtained from physical and simulated robots we show that the self-organized flocking of a swarm of robots can be effectively guided by a minority of informed robots within the flock. In our study, we use two metrics to measure the accuracy of the flock in following the desired direction, and the ability to stay cohesive meanwhile. Using these metrics, we show that the proposed behavior is scalable with respect to the flock’s size, and that the accuracy of guidance increases with 1) the “stubbornness” of the informed robots to align with the preferred direction, and 2) the ratio of the number of informed robots over the whole flock size. Hande C ¸ elikkanat KOVAN Lab., Dept. of Computer Eng., Middle East Technical University, Turkey, e-mail: [email protected] Ali Emre Turgut KOVAN Lab., Dept. of Computer Eng., Middle East Technical University, Turkey e-mail: [email protected] Erol S ¸ ahin KOVAN Lab., Dept. of Computer Eng., Middle East Technical University, Turkey e-mail: [email protected]

216

H. C ¸ elikkanat et al.

1 Introduction Swarm robotics takes its inspiration from natural swarms and aims to develop self-organization in large groups of robots with no centralized control while putting emphasis on flexibility, robustness and scalability. Most of the ongoing studies have focused on the application of self-organization approach. The limitations of controllability due to the use of the self-organization has been neglected so far, leaving the question of how useful the approach can be in real-world use, unanswered. In this study, we are interested in how, and to what extent we can control the behavior of a swarm robotic system. Specifically, we are interested in how behaviors that lead to self-organization in a robotic flock can be externally controlled. A number of studies investigated external control on animal swarms. In two interesting works, Vaughan et al. [17] used a robotic sheepdog to guide a duck flock to a predefined goal point, while Lien et al. [7] compared various approaching and steering strategies for external shepherds, utilizing them for herding, covering, patrolling, and collecting behaviors. In [8] they extended the results to the multiple shepherds case. Halloy et al. [4] manipulated the collective shelter selection process of cockroaches with robots that are socially integrated into the swarm. Meanwhile, the decision making mechanisms in natural flocks, and the possibility of their being guided by some of the individuals inside the flock are also investigated in various studies. Reebs [12] studied the decision making mechanisms in the foraging movements of fish schools, and showed that relatively few individuals with a priori knowledge can guide the whole school. Couzin et al. [2] modeled the decision making of flocks in which there are few informed individuals. Numerical simulations showed that the accuracy of the informed individuals in guiding the flock increases as the size of flock increases while the ratio of informed individuals is kept fixed. If the proportion of the informed individuals is moderate, increasing the importance given to the preferred direction increases the accuracy of the motion, however it also increases the fragmentation of the flock. The increase in the accuracy is not observed when the proportion of the informed individuals is either too small or too large. In [14], Shi et al. investigated the effect of informing some individuals in a flock with an external reference signal, from the control theory perspective, using point mass dynamics. They showed that stable motion, in which the agent velocities eventually converge to the desired velocity, can be achieved even if there is a single informed individual who could receive the signal, whereas increasing the number of informed individuals does not necessarily increase the convergence rate. Inspired by the works of [12], [2] and [14], in this paper we try to achieve a similar guidance mechanism in a robotic flock. Flocking in artificial swarms was first studied by Reynolds, who proposed a set of simple rules for obtaining realistic looking flying bird animations [13]. In robotics, Matari´c [11] was the first to achieve flocking in a collective homing behavior, composed

Guiding a Robot Flock via Informed Robots

217

of safe-wandering, aggregation, dispersion and homing behaviors. Kelly and Keating [9] developed a novel infrared (IR) system for the robots to sense the relative range and bearing of neighbors. The robots followed a leader which was elected by wireless communication. Hayes and Tabatabaei [5] proposed a leaderless distributed flocking algorithm, assuming that the robots could sense the range and bearing of their neighbors to compute the center of mass of the flock, which was used for cohesion and alignment. Although the algorithm was implemented successfully on the Webots simulator, in the physical experiments the sensors had to be emulated using an overhead camera. Holland et al. [6] proposed a flocking behavior for unmanned air vehicles based on avoidance, flock centering and alignment behaviors, with the range, bearing and velocity information received from a base station. Campo et al. [1] used a specifically designed colored LED system surrounding the body of sbots through which the s-bots negotiate their a priori estimations of the nest location, in order to collectively carry a prey. In [16], Turgut et al. presented a truly self-organized, leaderless, decentralized flocking in a robot swarm. In this study, we extend this behavior by informing some of the robots about the preferred direction that we wish the swarm to move. The informed robots do not signal that they are “informed”, and instead guide the rest of the swarm by their tendency to move in the preferred direction. We present experimental results on both physical and simulated robots, and show that the self-organized flocking of a swarm of robots can be effectively guided by a minority of informed robots within the flock, without an explicit leadership mechanism. Then we analyze the system’s performance under various conditions.

2 Experimental Platforms In this study, we use Kobot and its physics-based simulator CoSS [15]. Kobot is a light-weight (12 cm diameter), differentially driven robotic platform (Fig. 1(a)). It has two main sensory systems: the Infrared Short-Range Sensing System (IRSS) and the Virtual Heading Sensor (VHS). IRSS is composed of 8 infrared sensor modules located at 45◦ intervals around the base (Fig. 1(b)), and is used for short-range proximity measurements. It uses modulated infrared signals to minimize the environmental interference, and crosstalk among robots. The sensors can detect objects within a 21 cm range at seven discrete proximity levels, and can distinguish kin-robots from obstacles. The VHS is used for virtually “sensing” the relative headings of the neighboring robots. It consists of a digital compass and a wireless communication module. The robot’s heading with respect to the sensed North is measured using the compass module and is broadcasted to other robots through wireless communication. Each robot receives the broadcasted heading values of the robots within its communication range. The robots whose heading values can be received by a robot are called its VHS neighbors.

218

H. C ¸ elikkanat et al.

Y ns

vL y

ω

θ 3

2

0 5

( ) (a)

vR

1

4 6

u

7

(b)

l x

X

(c)

Fig. 1 (a) Photo of a Kobot. (b) Top-view of a Kobot sketch showing the body (circle), the IR sensors(small numbered rectangles), and the two wheels (gray rectangles). (c) The reference frame is fixed to the center of the robot where the x-axis coincides with the rotation axis of the wheels. The forward velocity (u) is along the y-axis. ω denotes the angular velocity of the robot. The y-axis of the bodyfixed reference frame makes an angle of θ with the sensed North direction (ns ) at the instant the figure is drawn, which is the current heading of the robot. With kind permission from Springer Science+Business Media: Swarm Intelligence, Selforganized flocking in mobile robot swarms, volume 2, number 2-3, 2008, Ali Emre Turgut, Hande C ¸ elikkanat, Fatih G¨ ok¸ce and Erol S ¸ ahin, Fig. 1

We would like to point out that VHS does not assume the sensing of absolute North. As a matter of fact, metal objects are abundant in indoor environments, and the sensed North deviates much from the absolute North direction. The only assumption that VHS makes is that the sensed North direction remains approximately the same among the VHS neighbors of a robot. In CoSS, the sensing and actuation of robots is modeled using data obtained from systematic experiments and is verified against Kobots [15]. The sensing characteristics of the IRSS is obtained from systematic experiments with Kobot. We model the noise on the VHS with the vectorial noise model [3]. A noise vector is added to the heading measurements performed by each robot. The direction of the noise vector is chosen from a Gaussian distribution, whose mean is the actual heading of the robot, and standard deviation is π/2. The magnitude of the noise vector, denoted by η, determines the amount of noise in the system, and is set to 1. The VHS module is modeled to receive heading information from 20 randomly chosen VHS neighbors within a range of 20 m at each control step.

3 Flocking Behavior The flocking behavior proposed in [16] originally consists of heading alignment and proximal control behaviors, to which we now add a preference for a certain direction:

Guiding a Robot Flock via Informed Robots

a=

219

h+β p+γ d h + β p + γ d

where h is the heading alignment vector, p is the proximal control vector, d is the direction preference vector, and a is the desired heading vector. β is the weight of the proximal control vector, and γ is the weight of the direction preference vector. The informed robots have a non-zero γ to include the preference vector d for the desired direction, whereas the naive robots have their γ set to zero, effectively discarding the direction preference term in their calculations. The heading alignment vector h tries to align the robot with its neighbors, and is calculated as:  iθ j j ∈N e h=   j∈N eiθj 

where N denotes the set of VHS neighbors, θj is the heading of the j th neighbor converted to the body-fixed reference frame and  ·  calculates the Euclidean norm. The proximal control behavior is responsible for flock cohesion and collision avoidance. The normalized proximal control vector p is calculated using the infrared readings from the IRSS. It is a vector sum of virtual forces which are assumed to act on each infrared sensor: 1 p= fkeiφk 8 k

where k ∈ {0, 1, · · · , 7} denotes the sensor positioned at angle φk= π 4 k with respect to the x-axis (see Fig. 1(b)), and fkdenote the virtual force acting on the sensor. The virtual forces are taken to be proportional to the square of the difference between the current detection level (ok) of the sensor, and the desired detection level (odes ). The desired detection level is defined as an intermediate detection level (3) if the sensed object is another robot and 0 if it is an obstacle. This setting motivates the robot to keep at an optimal distance from its peers and escape from obstacles. fk is then calculated as follows:  2 des ) if ok ≥ odes − (ok −o C2 fk = (ok −odes ) otherwise. C The direction preference vector d is calculated as: d = dp− ac where ac is the current heading vector of the robot coincident with the yaxis of the body-fixed reference frame (see Fig. 1(c)), and dp stands for the preferred direction. The desired heading vector, a, is used to calculate the forward (u) and angular (ω) velocities. u is calculated via diminishing the maximum speed

220

H. C ¸ elikkanat et al.

umaxaccording to the robot’s momentary urge to turn. This urgency is given by the difference between the desired heading and the current heading of the robot, calculated by a dot product of the two vectors: u=



(a · ac ) umaxif a · ac ≥ 0 0 otherwise

The angular velocity ω is controlled by a proportional controller: ω=

1 ( ac −  a) 2

4 Metrics We evaluate the performance of the system using two different metrics. The accuracy metric, adopted from [2], measures the angular deviation of the direction of the flock from the preferred direction. The angular deviation is analogous to the standard deviation in linear statistics for inherently directional data. For a given set of vectors, it is calculated as: ¯ cos(x¯0 − α) C¯ = R  ¯ S = 2(1 − C)

¯ is the length of the mean vector of the given vectors, x¯0 is the where R direction of the mean vector, α is the preferred direction and S is the angular deviation around this preferred direction [10]. Then, the accuracy is defined as: Accuracy = 1 − S/2 The accuracy metric becomes 1 when the angular deviation is minimum, and 0 when the angular deviation is maximum. In this study, the angular deviation is calculated for the direction of motion of the flock center in all experiments. The direction is calculated for the final period of each experiment, discarding the transient phase in which a common direction has not yet settled. The cohesiveness metric uses the size of the largest flock within the environment to evaluate the degree of cohesion. We assumed that a robot is part of a flock if it is within the infrared range of another robot that already belongs to the flock. This metric is important since the infrared sensors have a limited range and that once a robot gets disconnected from the rest of the flock, it has little chance to find it again.

180

180

90

90

heading [degree]

heading [degree]

Guiding a Robot Flock via Informed Robots

0

−90

−180 0

221

0

−90

50

100 time [s]

150

200

(a)

−180 0

50

100 time [s]

150

200

(b)

Fig. 2 Time evolution of the headings of 100 robots in two sample experiments. 10 robots are commanded to go in 90◦ direction. The traces of informed robots are indicated with white bold lines. (a) γ = 0.5 (b) γ = 1

5 Experimental Results In this section, we exploit the flocking behavior discussed above to analyze the effect of informing a subset of robots about a preferred direction of motion. We perform two different analyses: 1) the effect of the flock size for varying ratios ρ of the number of informed robots over the whole flock size, and 2) the effect of the weight of the preferred direction (γ) for varying ratios ρ. The experiments are conducted with both physical and simulated robots. In the physical experiments 7 Kobots are used, whereas the experiments conducted in CoSS use 10, 20 or 100 robots. The weight of the proximal control behavior (β) is set to 4, while odes is set to 3 for kin-robots and 0 for obstacles, and umax = 7 cm/s. The VHS noise magnitude η is set to 1 in the simulations. The experiments are conducted for 60 s for physical robots, and 1000 s in the simulations. Physical experiments are repeated for 5 times, the simulations for 100 times. The direction of motion of the flock center is calculated during the last 40 s of the experiments with the physical robots, and during the last 125 s with the simulated ones. The robots are initialized with random orientations, and the informed robots are assigned randomly. Fig. 2 plots the time evolution of the headings of 100 robots in two sample experiments. There are 10 informed robots (indicated with white traces) which are commanded to go in 90◦ direction, and their γ is set to either 0.5 or 1. In both cases, the directions of the robots fluctuate until they consent on the preferred direction. In the γ = 0.5 case, the alignment is reached faster (Fig. 2(a)), however in the γ = 1 case, the informed robots are more “stubborn” to follow the preferred direction, so the alignment is reached only when the whole flock consents on the preferred direction (Fig. 2(b)). Flock Size Experiment: In this experiment, we investigate the effect of the size of flock on the accuracy of flock direction. We vary the size of the

222 1 0.9 0.8 0.7 accuracy

Fig. 3 Plot of accuracy of flock direction as function of ratio of informed robots for different flock sizes. The error bars are not shown since accuracy is a function of angular deviation

H. C ¸ elikkanat et al.

0.6 0.5 0.4 0.3 CoSS, 100 robots CoSS, 20 robots CoSS, 10 robots 7 Kobots

0.2 0.1 0 0

0.2

0.4

0.6

0.8

1

ρ

flock (10, 20 and 100 in simulations and 7 in physical experiments) and measure accuracy for different ratios ρ of the number of informed robots. In the experiments, γ is set to 0.1 and the results are plotted in Fig. 3. Fig. 3 shows that the accuracy is independent of the flock size for a fixed ratio of informed robots, that is, the proposed behavior is scalable. It is also observed in the figure that for a fixed system size, increasing the ratio increases the accuracy of the direction of motion asymptotically. Moreover, the accuracy is quite high for even low ratios, and the increase in the accuracy is very rapid with respect to the increase in ρ. The results of the Kobot experiments are slightly less accurate due to the limited test area used for the experiments, since the system may not reach to steady-state in 60 s especially for low ratios, however the trends are similar. Weight of the Preferred Direction Experiment: In this experiment, we investigate the effect of the weight of the preferred direction, γ, on the accuracy of flocking motion. We vary γ and measure accuracy for different ratios ρ. We also measure the size of the largest flock eventually formed for various γ and ρ. 100 and 7 robots are used in simulations and Kobot experiments, respectively. The results are plotted in Fig. 4(a) and 4(b). It is observed in Fig. 4(a) that γ has an effect on the accuracy of motion for moderately low ratios (ρ = 0.1 and ρ = 1/7). For the high ratios, the accuracy stays flat at approximately 1 irrespective to γ. Likewise, for very low ratios, the accuracy does not increase with γ. The results of the Kobot experiments are again slightly less accurate due to the limited test area, but the trends are the same. In Fig. 4(b) it is seen that for the intermediate ratios, increase in γ decreases the size of the largest flock and therefore results in increased fragmentation of the flock, which is not observed in low or high ratios (ρ = 0.01 and ρ = 0.8). The reason of this phenomenon is that the more “stubborn” are the informed robots to move in their preference, the more probable that they will be separated from the flock. Therefore, in the ρ = 0.01 case, the loss of a single informed robot does not have a significant effect on the largest flock size. On the other hand, when the ratio is high enough (ρ = 0.8, ρ = 1.0),

1

100

0.9

90

0.8

80

0.7

70

size of largest flock

accuracy

Guiding a Robot Flock via Informed Robots

0.6 0.5 0.4 0.3 CoSS, ρ=1/100 CoSS, ρ=10/100 CoSS, ρ=50/100 CoSS, ρ=100/100 7 Kobots, ρ=1/7

0.2 0.1 0 0

0.2

0.4

0.6

0.8

223

60 50 40 30 γ=0.1

20

γ=0.5

10 0 1

γ=1 0

0.01

0.1

0.8

1

ρ

γ

(a)

0.5

(b)

Fig. 4 (a) Plot of the accuracy of flock direction as a function of the weight of the preferred direction (γ) for different ratios of informed robots (ρ). (b) Plot of the size of the largest flock as a function of ρ for varying γ. The ends of the boxes and the horizontal line in between correspond to the first and third quartiles and the median values, respectively. The top and bottom whiskers indicate the largest and smallest non-outlier data, respectively. The data in between the first and third quartiles lie within the 50% confidence interval, while the data in between the whiskers lie within the 99.3% confidence interval

the informed robots are faster in changing the direction of the whole flock, which reduces the length of the transition phase, and decreases separations.

6 Conclusion In this study, we showed that the self-organized flocking motion in a robot swarm can be guided by informing a subset of the robots to prefer a certain direction of motion. The results show that, in the presence of even a small number of informed robots, the flock can consent on the preferred direction of motion, and furthermore, the proposed behavior is scalable with respect to the flock’s size. The directional guidance of the flock becomes more accurate with 1) increasing the “stubbornness” of the informed robots, and 2) increasing the number of informed robots. We have also showed that, for moderately low ratios of informed robots, increasing the importance given to the preferred direction has the adverse effect of increasing the fragmentations in the group. Among the work that awaits to be done, there is the analysis of the transient dynamics of the system, control and reduction of fragmentation, and the analysis of sensitivity against the VHS noise, as well as the spatial locations of informed robots in terms of accuracy and cohesiveness. Acknowledgements. The authors express their gratitude for the most valuable comments of Emre U˘ gur, KOVAN Research Lab, Department of Computer Engineering, Middle East Technical University, and Onur Soysal, Department of

224

H. C ¸ elikkanat et al.

Computer Science and Engineering, University at Buffalo, SUNY. Hande C ¸ elikkanat ¨ ITAK ˙ acknowledges the partial support of the TUB (Turkish Scientific and Technical Council) graduate student research grant. The work of Ali Emre Turgut is ¨ ITAK ˙ supported by TUB under grant no: 104E066. This work has been carried out ¨ ITAK ˙ ˙ in TUB “KARIYER: Kontrol Edilebilir Robot O˘ gulları” project with number 104E066. The simulations are run on the High Performance Computing Cluster of the Department of Computer Engineering, Middle East Technical University.

References 1. Campo, A., Nouyan, S., Birattari, M., Groß, R., Dorigo, M.: Negotiation of goal direction for cooperative transport. In: Proc. of the 18th Belgium-Netherlands Conference on Artificial Intelligence, pp. 365–376. University of Namur, Namur (2006) 2. Couzin, I.D., Krause, J., Franks, N.R., Levin, S.A.: Effective Leadership and Decision-Making in Animal Groups on the Move. Nature 433, 513–516 (2005) 3. Gregoire, G., Chate, H., Tu, Y.: Moving and staying together without a leader. Physica D 181, 157–170 (2003) 4. Halloy, J., Sempo, G., Caprari, G., Rivault, C., Asadpour, M., Tache, F., Said, I., Durier, V., Canonge, S., Ame, J.M., Detrain, C., Correll, N., Martinoli, A., Mondada, F., Siegwart, R., Deneubourg, J.L.: Social integration of robots into groups of cockroaches to control self-organized choices. Science 318, 1155–1158 (2007) 5. Hayes, A.T., Dormiani-Tabatabaei, P.: Self-organized flocking with agent failure: Off-line optimization and demonstration with real robots. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 3900– 3905. IEEE Press, New Jersey (2002) 6. Holland, O., Woods, J., Nardi, R., Clark, A.: Beyond Swarm Intelligence: the UltraSwarm. In: Proceedings of the IEEE Swarm Intelligence Symposium, pp. 217–224. IEEE Press, New Jersey (2005) 7. Lien, J.-M., Bayazit, O.B., Sowell, R.T., Rodr´ıguez, S., Amato, N.M.: Shepherding behaviors. In: Proceedings of the IEEE Conference on Robotics and Automation, pp. 4159–4164. IEEE Press, New Jersey (2004) 8. Lien, J.-M., Rodr´ıguez, S., Malric, J.-P., Amato, N.M.: Shepherding behaviors with multiple shepherds. In: Proceedings of the IEEE Conference on Robotics and Automation, pp. 4159–4164. IEEE Press, New Jersey (2004) 9. Kelly, I.D., Keating, D.A.: Flocking by the fusion of sonar and active infrared sensors on physical autonomous robots. In: Proceedings of the Third International Conference on Mechatronics and Machine Vision in Practice, vol. 1, pp. 14–17. Universidade do Minho, Guimar˜ aes (1996) 10. Mardia, K.V.: Statistics of Directional Data. Academic Press, London (1972) 11. Matari´c, M.J.: Interaction and intelligent behavior. Ph.D. thesis, MIT (1994) 12. Reebs, S.G.: Can a minority of informed leaders determine the foraging movements of a fish shoal? Animal Behaviour 59, 403–409 (2000) 13. Reynolds, C.W.: Flocks, herds and schools: A distributed behavioral model. In: Stone, M.C. (ed.) Proceedings of the 14th annual conference on computer graphics and interactive techniques (SIGGRAPH 1987), pp. 25–34. ACM Press, New York (1987)

Guiding a Robot Flock via Informed Robots

225

14. Shi, H., Wang, L., Chu, T.: Virtual Leader Approach to Coordinated Control of Multiple mobile Agents with Asymmetric Interactions. Physica D 213, 51–65 (2006) 15. Turgut, A.E., C ¸ elikkanat, H., G¨ ok¸ce, F., S ¸ ahin, E.: Self-Organized Flocking in Mobile Robot Swarms. Swarm Intelligence 2(2-3) (2008) 16. Turgut, A.E., C ¸ elikkanat, H., G¨ ok¸ce, F., S ¸ ahin, E.: Self-Organized Flocking with a Mobile Robot Swarm. In: Padgham, Parkes, M¨ uller, Parsons (eds.) Proceedings of the 7th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2008), pp. 39–46. International Foundation for Autonomous Agents and Multiagent Systems, South Carolina, SC (2008) 17. Vaughan, R., Sumpter, N., Henderson, J., Frost, A., Cameraon, S.: Experiments in Automatic Flock Control. Robotics and Autonomous Systems 31, 109–117 (2000)

Theoretical and Empirical Study of Pedestrian Outflow through an Exit Daichi Yanagisawa, Ayako Kimura, Ryosuke Nishi, Akiyasu Tomoeda, and Katsuhiro Nishinari

Abstract. In this paper, we have detailedly studied the factors of increasing and decreasing the pedestrian outflow through an exit. One of the major factors is a conflict. In the floor field model, which is a pedestrian model using cellular automata, the conflicts are taken into account by the friction parameter. However, the friction parameter is a constant and does not depend on the number of the pedestrians conflicting at the same time. We have extended the friction parameter to the friction function, which is a function of the number of the pedestrians involved in the conflict. Furthermore, we also consider the effect of turning around at the exit and the effect of avoiding conflicts by going through the exit one after the other, i.e., zipper effect. The results of theoretical analysis of the extended model, which includes three new effects, agree with the experimental results much better than the previous model. We have also found that putting an obstacle in front of the exit increases the pedestrian outflow from our experiments. The friction function clearly explains the mechanism of the effect of the obstacle, i.e., the obstacle blocks a pedestrian moving to the exit and decreases the average number of pedestrians involved in the conflict. Daichi Yanagisawa, Ayako Kimura, Ryosuke Nishi, Akiyasu Tomoeda, and Katsuhiro Nishinari Department of Aeronautics and Astronautics, School of Engineering, The University of Tokyo. 7-3-1, Hongo, Bunkyo-ku, Tokyo 113-8656, Japan Daichi Yanagisawa Research Fellow of the Japan Society for the Promotion of Science. 5-3-1, Kojimachi, Chiyoda-ku, Tokyo 102-8471, Japan Katsuhiro Nishinari PRESTO, Japan Science and Technology Agency. Sanbanmachi-Building, 5, Sanbanmachi, Chiyoda-ku, Tokyo 102-0075, Japan e-mail: {tt087068,tt076360,tt076379,tt67055,tknishi} @mail.ecc.u-tokyo.ac.jp

228

D. Yanagisawa et al.

1 Introduction Pedestrian Dynamics has been studied vigorously over last decades [1, 2]. Many microscopic models are developed such as the floor field (FF) model [3, 4, 5], the social force model [6], and the multi-grid model [7], to simulate a crowd of pedestrians movement realistically. In addition to the simulations, there are also many pedestrian experiments [8] to study collective behaviors of pedestrians. The behavior of pedestrians at an exit is focused on by many researchers as that of the multi-robot system’s [9] is, since it greatly affects the total evacuation time in an emergency situation [5, 8]. One of the major phenomenon observed at an exit is a conflict, which occurs when more than one pedestrian move to the same place at the same time. When many conflicts occur, the pedestrian outflow, which is the number of pedestrians going through an exit per unit time, decreases. In the floor field model, which is a pedestrian dynamics model using cellular automata, conflicts are taken account by friction parameter µ ∈ [0, 1]. It describes clogging and sticking effects between pedestrians, and in a conflict situation, the movement of all involved pedestrians is denied with probability µ . Since µ is a constant parameter, the strength of clogging and sticking does not depend on the number of pedestrians involved in the conflict. In reality, however, it is more difficult to avoid a conflict when three pedestrians move to the same place at the same time than two pedestrians move to the same place at the same time. Therefore, we newly introduced the friction function, which is a function of the number of pedestrians involved in the conflict. We also introduce the effect of turning around and the effect of going through the exit one after the other, i.e., zipper effect. When pedestrians turn around, their walking speed decreases, and when pedestrians try to go through the exit one after the other, they can avoid conflicts. These extensions of the floor field model make it possible to describe the pedestrian behavior around an exit more precisely and realistic, and the outflow obtained by the extended model agrees with the experimental data well. We have also found that we can increase the pedestrian outflow by putting an obstacle in front of the exit from our experiments. The mechanism of the effect of the obstacle is difficult to explain by friction parameter, however, we have successfully explained it by the friction function.

2 Floor Field Model We consider a situation that every pedestrian in a room moves to the same exit. In the floor field model, the room is divided into cells as given in Fig. 1 (a). Man shaped silhouettes represent pedestrians, an alphabet E and alphabets O represent the exit cell and obstacle cells, respectively. Each cell contains only a single pedestrian at most. Every time step pedestrians choose which cell to move from 5 cells: a cell which the pedestrian stands now and the four Neumann neighboring cells (Fig. 1(b)). Two kinds of FFs determine the probability of which direction to move, i.e., Static FF, which is the shortest distance to the exit cell, and Dynamic FF, which is a

Theoretical and Empirical Study of Pedestrian Outflow through an Exit E

229

p0,1

p−1,0 p0,0 p1,0 O O O

O O O

p0,−1

t+1

t

(b)

(a)

Fig. 1 (a) A schematic view of an evacuation simulation by the FF model. Pedestrians proceed to the exit by one cell at most by one time step. (b) Target cells for a pedestrian at the next time step. The motion is restricted to the Neumann neighborhood in this model

µ E E

1− µ or Fig. 2 The way of solving conflicts. In a conflict situation, movement of all involved pedestrians remain at their cell with probability µ . One of them is randomly allowed to move to the desired cell with probability 1 − µ

number of footprints left by the pedestrians [4]. In the following, we consider only the effect of Static FF, for simplicity. Due to the use of parallel dynamics it happens that two or more pedestrians choose the same target cell in the update procedure. Such situations are called conflicts in this paper. To describe the dynamics of a conflict in a quantitative way, friction parameter µ ∈ [0, 1] was introduced in Ref. [3]. This parameter describes clogging and sticking effects between the pedestrians. In a conflict the movement of all involved pedestrians is denied with probability µ , i.e., all pedestrians remain at their cell. Therefore, the conflict is solved with probability 1 − µ , and one of the pedestrians is allowed to move to the desired cell (Fig. 2). The pedestrian which actually moves is then chosen randomly with equal probability. We extend the floor field model to focus on the pedestrians of behavior at the exit (Fig. 3 (a)). The normal floor field model, which is explained above, uses the Neumann neighboring cells, however, we consider the situation that there are n neighboring cells of the exit cell and k pedestrians are trying to move to the exit cell. The case n = 3 and n = 5 corresponds to the FF model using the Neumann neighborhood and Moore neighborhood, respectively. The transition probability to the exit

230

D. Yanagisawa et al.

α β

1

E

µi β

E

1.0 0

0.8

1 2

theo. theo. theo.

0 1 2

0.5 0.48 0.60

0.6

k

0.4

β 1− β

0.2

2

n pedestrians

1− β

1 − µi

0.0

0

1

2

3

4

5

k

(a)

(b)

Fig. 3 (a) Schematic view of the extended model at the exit cell. α and β represent the transition probability. µi is the friction parameter or friction function, and k is the number of pedestrians trying to move to the exit cell. (b) Values of the friction parameter and the friction functions against k, which is a number of pedestrians who move to the exit cell at the same

cell determined by the Static FF is represented as β ∈ [0, 1] for simplicity. α is a probability of getting out from the exit.

3 Friction Function The friction parameter, which represents the strength of clogging and sticking between pedestrians, is a constant and does not take account of the difference of the number of the pedestrians involved in the conflict. However, the clogging effect is stronger when three pedestrians conflict each other than when two pedestrians conflict each other. Therefore, we newly introduce the friction function µ1 and µ2 , which are functions of k ∈ N, i.e., the number of pedestrians moving to the exit cell at the same time. µ1 and µ2 are described as follows:

µ1 (ζ1 , k) = 1 − exp[−ζ1 (k − 1)] µ2 (ζ2 , k) = 1 − (1 − ζ2)k − kζ2 (1 − ζ2)k−1 ,

(1) (2)

and both µ1 and µ2 satisfy the conditions of friction functions: µi (ζi , 1) = 0, µi (ζi , ∞) = 1 (i = 1, 2). The former equations mean that there is no conflict when only one pedestrian move to the exit cell, and the latter means that no one can move to the exit cell when greatly many pedestrians move to there at the same time. µ1 is one of the simplest form of friction functions, which satisfies the conditions of them. We decide to use an exponential function since it is used in the transition probability in the FF model [3, 4, 5]. An exponential function is also easy to analyze theoretically. ζ1 ∈ [0, ∞) is friction coefficient, which represent the strength of the clogging irrelevant to k. When ζ1 increases, µ1 increases. We define µ2 by considering the psychological effect of pedestrians. ζ2 ∈ [0, 1] is a probability of not giving way to others when more than one pedestrian move to the exit cell at the same time. The second term in (2) is the probability that every pedestrian involved in the conflict tries to give way to others. The third term is the

Theoretical and Empirical Study of Pedestrian Outflow through an Exit

231

probability that only one pedestrian does not give way to others while the others do. By subtracting the two terms, which are the probabilities of resolving a conflict, we obtain the friction function µ2 . When ζ2 increases, µ2 increases. To deal in the friction parameter in the same way as the friction functions, we describe friction parameter as µ0 in the following. The explicit form of friction parameter µ0 is described as follows by using k, which is the number of pedestrians involved in the conflict.  0 (k = 1) (3) µ0 = ζ0 (k ≥ 2), where ζ0 is a value of friction parameter, when a conflict occurs. Figure 3 (b) is the plots of µ0 , µ1 , and µ2 as a function of k. µ0 is constant in k ≥ 2, while µ1 and µ2 gradually increase as k increases and reflect the difference of the strength of clogging against k. The friction functions and the friction parameter are used to study clogging effect at bottlenecks. The friction parameter is applied to calculate an outflow from a wide exit in Ref. [5]. They are also effective for considering outflows from plural exits.

4 Effect of Turning around at an Exit Pedestrians, who move to the exit cell from lateral direction as in Fig. 4 (a), have to turn their face to the outside of the room at the exit cell to get out from the room. When they turn around, their walking speed decreases, thus, we introduce this effect to the model by determining α , which is the probability of getting out from the room, as α = β cos (aθ ) . (4)

θ is the angle of incidence as in Fig. 4 (a), and a ∈ [0, 1] is a parameter, which describes the strength of decreasing walking speed by turning around. When a = 0, there is no decrease in walking speed by turning around, i.e., pedestrians walk in the same speed whether they are walking straightly or turning around. When a = 1, the decrease in walking speed is maximum. Pedestrians lose all velocity perpendicular to the exit direction. We consider the situation that pedestrians turn around at the exit cell. If they turn around gradually as they walk near to the exit, their walking speed do not decrease as much as the case we consider, and the expression of the α becomes different from (4).

5 Zipper Effect When many pedestrians egress through the narrow exit, we often see that pedestrians in the left area and pedestrians in the right area go through the exit one after the other. This phenomenon is called zipper effect [8], and when it occurs, the outflow

232

D. Yanagisawa et al.

Exit Direction

E

E

E

B2

θ= θ=

E A2

π

C2 B1

2

A1

C1

π 6

(a)

(b)

(c)

(d)

Fig. 4 (a) Schematic view of pedestrians’ angle of incidence. (b) Schematic view of twoparallel-line egress process. Pedestrians move to the exit cell one after the other. (c) Schematic view of normal egress process. It is difficult to determine the sequence of the evacuation. (d) Schematic view of two-parallel-line egress process with cells. First, pedestrians try to move to the dashed-line cell B1. If they conflict at the cell B1, they proceed to A1 and C1, and try to to move to the dashed-line cell B2. If they conflict again, they proceed to A2 and C2, and try to move to the exit cell E. Therefore, they have three chances to join into one line without stopping

increases since the number of conflicts between pedestrians deceases dramatically. It is also studied in highway traffic dynamics [11]. The zipper effect clearly observed in the case that pedestrians in the two parallel lines join into one line (Fig. 4 (b)) since there are only two pedestrians who can move to the exit cell. In the normal egress process as in Fig. 4 (c), there are more than two pedestrians who can move to the exit cell, and pedestrians cannot easily decide whether to go or give way to the others. Therefore, the sequence of the evacuation is more difficult to determine than the two-parallel-line case. If there is some distance to the exit cell, pedestrians in the two-parallel-line try to join into the one line before they arrive at the exit cell since they know the exit is narrow for two pedestrians going through at the same time. We model this phenomenon by increasing the chance of joining into one line. In the normal evacuation situation there is only one chance to move to the exit cell without stopping by avoiding a conflict. However, in the two-parallel-line case, there are m ∈ [1, ∞) chances as in Fig. 4 (d). If they conflict at the first chance, they proceed to one cell to the exit instead of stopping at their cell.

6 Average Pedestrian Outflow through an Exit In this section, the average pedestrian outflows through an exit are theoretically calculated. The probability of k pedestrians trying to move to the exit cell is   n bk (β ) = β k (1 − β )n−k , (5) k

Theoretical and Empirical Study of Pedestrian Outflow through an Exit

233

and by using this, the probability of one pedestrian succeeds to move to the exit cell is described as n

r(β , ζi , n) =

∑ [(1 − µi(ζi , k))bk (β )]

(i = 0, 1, 2).

(6)

k=1

We define pt (0) as the probability that a pedestrian is not at the exit cell at time step t and pt ( j) (1 ≤ j ≤ n) as the probability that a pedestrian j is at the exit cell at time step t. The pedestrian j is a pedestrian who was at the j th of n neighboring cells of the exit in the previous step. If greatly many pedestrians are gathering around the exit, i.e., there are always n ∈ N pedestrians at the neighboring cells of the exit cell, the master equation is described as follows: ⎡ ⎤ ⎡ ⎤ ⎤⎡ {1 − r(β , ζi , n)}m α (θ1 ) . . . . . . α (θn ) pt+1 (0) pt (0) ⎢ pt+1 (1)⎥ ⎢ 1 [1 − {1 − r(β , ζi , n)}m ] 1 − α (θ1) ⎥ ⎢ pt (1)⎥ ⎢ ⎥ ⎢n ⎥ ⎥⎢ ⎢ .. ⎥ = ⎢ ⎥ ⎢ .. ⎥ , .. . . ⎣ . ⎦ ⎣ ⎣ ⎦ . . . ⎦ m 1 pt+1 (n) pt (n) 1 − α (θn ) n [1 − {1 − r(β , ζi , n)} ] (7) where θ j is the incident angle of the pedestrian who was at the j th neighboring cell, and m represents the number of chances to join into the one line in two-parallel-line case. If we substitute m = 1 to (7), we obtain a master equation which consider the confluence only at the exit cell. From (7) with the normalization condition pt (0) + ∑nj=1 pt ( j) = 1, we obtain the average pedestrian outflow through an exit in the stationary situation, i.e., t → ∞. The mathmatical expression of the pedestrian outflow is complicated, however, in the case α is constant, i.e., the effect of turing is neglected, it is simply described as

0

n

q(β , α , ζi , n, m) = α

α (1 − {1 − r(β , ζi, n)}m )

0

∑ pt ( j) = α + (1 − {1 − r(β , ζi, n)}m )

(i = 0, 1, 2). (8)

j=1

7 Pedestrian Outflows of the Experiment and the Theory Here, we compare the pedestrian outflows from the experiments and the theoretical analysis on the extended model. We did the evacuation experiments to verify the relation between the three new factors and the average pedestrian outflow q. The width of the exit is as wide as the breadth of the participants’ shoulders, i.e., 50 cm. There are 18 participants of the experiments, who are all men. The experiments started when we clapped our hands and finished when the all participants evacuated from the room. Nine kinds of initial conditions are put into practice (Fig. 5). In the case (H) and (I) pedestrians could move as they want after the evacuation started, however, in the case (A) to (G) pedestrians had to follow the former pedestrian, i.e., they were prohibited from putting the queues into disorder.

234

D. Yanagisawa et al.

π

150 cm

0

π

(A) n=1

(B) n=2

0

2

(C) n=2

6

π 2

(D) n=2

4

0

(E) n=3

2 0

(F) n=3

150 cm

π 2

π

π

π 6

(G) n=4

(H)

(I)

Fig. 5 Schematic views of nine initial conditions of the experiments. The figures described by the silhouettes of pedestrians (0, π /6, π /4, π /2) are the approximated incident angles used in theoretical calculation. The participants of the experiments are not exactly go to the exit along the angle of incidence described in the figures

The results of the experiments are described in Tab. 1. There are four remarkable points in this table. First, we see that the pedestrian outflow decreases in n ≥ 2. This indicates that when the number of pedestrians at the neighboring cells of the exit increases, the outflow decreases. Second, the outflow decreases with increasing of θ , when n is constant. Third, the maximum outflow is attained in the case (B), i.e., two-parallel-line case. See Fig. 7 (a), which is a snapshot of the experiment (B). We clearly see the zipper effect, i.e., pedestrians in the left line and the right line go through the exit one after the other. Pedestrians are intelligent enough to avoid conflicts by their selves. Forth, the pedestrian outflow in the case (G) (n = 4) and the average of pedestrian outflow in the case (H) and (I) (Tab. 1 (H&I)) are similar. This indicates that there are approximately four pedestrians at the exit in the normal evacuation situation when the width of the exit is 50 cm. The parameters in the theory are calculated in the following way. First, we define the cell size as 50×50 [cm2 ] and 1 time step as 0.3 [sec] and decide the β from the result of the experiment (A). In the experiment (A) there is no conflict, thus we obtain the value of β independent from the kinds of the friction functions. Next,

Theoretical and Empirical Study of Pedestrian Outflow through an Exit

235

Table 1 The pedestrian outflows of the experiments and theoretical analysis. “Case” in the table is corresponding to the case in Fig. 5. The pedestrian outflow in the case (H&I) represents the average pedestrian outflow of the case (H) and (I). n is the number of pedestrians at the neighboring cells of the exit cell. (θ )i [rad] is the approximated angle of incidence, and i is number of pedestrians move to the exit along the direction of θ . The numbers of the experiments N are described in the forth line of the table. The pedestrian outflows qexp [person/(m· s)] in the table are the average of the N experiments Case (A) (B) (C) (D) (E) (F) (G) (H) (I) (H&I) n 1 2 2 2 3 3 4 θ (0)1 ( π6 )2 (0)1 , ( π2 )1 ( π2 )2 (0)1 , ( π4 )2 (0)1 , ( π2 )2 ( π6 )2 , ( π2 )2 N 3 3 3 3 2 3 2 2 3 qexp 2.65 2.94 2.75 2.60 2.68 2.59 2.47 2.44 2.52 2.48

Pedestrian Outfolw

3.2

A1 B2 C2 D2 E3 F3 G4 H&I

3.0

  2.8 2.6 2.4

 

Bar Experiments  Friction Parameter: Μ0  Friction Function: Μ1  Friction Function: Μ2

               

A1 B2 C2 D2 E3 F3 G4 H&I

Kind of Experiment n Fig. 6 Pedestrian outflows of the experiments and the theoretical calculation using µ0 , µ1 , and µ2 in the case a = 1/3. The numbers in the parentheses of the tick of the horizontal axis represent n. The results of the µ1 and µ2 agree with those of the experiments well

we calculate the friction coefficient ζi (i = 0, 1, 2) by using the value of (H&I) in Tab. 1. The experiments (A) to (G) are far from the real evacuation since the participants are forbidden from putting the queues into disorder; however, (H) and (I) are normal evacuation. Thus, we use the data of (H&I) to obtain the ζi in the normal evacuation situation. In the above paragraph, we found that there are approximately four pedestrians at the exit in the normal evacuation situation. Therefore, we decide n = 4 in the case (H&I). We obtain the value of β , ζi (i = 0, 1, 2) as follows: β = 0.80, ζ0 = 0.251, ζ1 = 0.130, ζ2 = 0.300. In the case (B), i.e., two-parallelline case, we calculate m = 3 from “Distance from the exit” / “Length of a cell” = 150 [cm] / 50 [cm]. In the other cases, m = 1. The pedestrian outflows of the theoretical analysis are compared with those of experiments in Fig. 6. We see that the results of the friction functions agree with those of the experiments very well, however, the results of the friction parameter does not. The value of the friction parameter is constant in k ≥ 2. Thus, the friction

236

D. Yanagisawa et al.

parameter, which is decided by using the experimental result in the normal evacuation situation, i.e., n = 4, is large for the case n = 2 and n = 3. The pedestrian outflow of the friction parameter is small for this reason. Since the friction functions consider the difference of n, they reproduce the value of the experiments. We would also like to mention that the difference of the outflow of the experiments (B), (C), and (D) are reproduce properly since we consider the effect of turning around and the zipper effect. The number of the pedestrians involved in a conflict is same in the case (B), (C), and (D), thus we obtain the same value from the theoretical calculation when we only considered the effect of the friction functions. Introducing the effect of turning around makes it possible to reflect the difference among (B), (C), and (D). The outflow of the case (B), which is calculated theoretically without the zipper effect, is much smaller than that of experiment. Therefore, the zipper effect is also necessary for the model.

8 The Effect of an Obstacle We study the effect of an obstacle put in front of the exit in this section. We did the evacuation experiments at the NHK TV studio in Japan. Two large walls were set up in the studio, and we could adjust the width of the exit by moving them (Fig. 7 (b1)). We decided it as 50 cm. The participants of the experiment were fifty women, who were their thirties and forties. We did three kinds of experiments in Fig. 7 (b). The experiment (b1) was the evacuation in a line, which was the same as the experiment (A) in Fig. 5 (A). The experiment (b2) was the normal evacuation, whose initial condition was the same as the experiment (I) in Fig. 5 (I). The initial condition and the way of the evacuation in the experiment (b3) was the same as the experiment (b2), however, the column, whose diameter was 20 cm, was put in front of the exit. The pedestrian outflows of the three cases are described in Tab. 2. The outflow in the case (b1) is the largest, since there was no conflict. Comparing the result of the experiment (b2) and (b3), we surprisingly find that the outflow of the experiment (b3), i.e., the experiment putting an obstacle in front of the exit, is larger than the experiment (b2), which is a normal evacuation. We explain this phenomenon by our theory. Looking the video of the experiment, we see that the obstacle blocks the participant moving to the exit. In the Sec. 7, we found that in the normal evacuation there are approximately four pedestrians at the exit, i.e., n = 4 (Fig. 7 (b2)). When the obstacle is put in front of the exit, there are approximately three pedestrians at the exit, since it blocks one pedestrian moving to the exit. Therefore, we consider that n = 3 in the experiment (b3) (Fig. 7 (b3)). We decide β from the experiment (b1) and ζi (i = 0, 1, 2) from the experiment (b2). The parameters are set to a = 1/3, m = 1, and the three incident angles are set to (0, π /2, 1.01) since the obstacle is shifted 25 [cm] from the center of the exit. Then we calculate the pedestrian outflow in the case experiment (b3), i.e., q(n = 3). We obtain β = 0.961 and the other results as in Tab. 3. We see that the pedestrian outflows calculated by using the friction functions µ1 and µ2 agree with that of the experiment (b3) in Fig. 7 (b3) well again, while the pedestrian outflow calculated by

Theoretical and Empirical Study of Pedestrian Outflow through an Exit 50 cm 50 cm

wall

237

65 cm

obstacle( =20cm)

(b1)

(a)

(b2)

(b3)

Fig. 7 (a) Snapshot of the experiment (B) (n = 2). The pedestrians in the left queue and the right queue go through the exit one after the other. (b1), (b2), and (b3) are the schematic views of the experiments in the case ”Going through the exit in a line”, Normal evacuation, and Putting an obstacle in front of the exit, respectively Table 2 The result of the experiments. We did (b1) three times, and (b2) and (b3) six times. The values are the average of the each experiment Experiment (a)Going through the exit in a line (b)Normal evacuation (c)Putting an obstacle in front of the exit

Pedestrian Outflow q [person/(m· s)] 3.20 2.75 2.93

Table 3 Analytical calculation in the case experiment (b3) Kind of µ µ0 µ1 µ2

ζi 0.230 0.092 0.240

Pedestrian Outflow q [person/(m· s)] 2.76 2.89 2.93

using the friction parameter µ0 is much smaller. This result justifies our assumption that the obstacle increase the pedestrian outflow since it decreases n, which is the number of pedestrians at the neighboring cells of the exit cell.

9 Conclusion In this paper we introduce the friction function, which changes its value against the number of pedestrians involved in a conflict. By using the friction functions, we obtain the more realistic figure of the pedestrian outflow through an exit, which corresponds to the result of the experiments very well. The pedestrian outflow using friction parameter does not agree with the experimental result well, therefore, the introduction of the friction function is necessary. The effect of turning around and the zipper effect are also considered to reproduce the pedestrian outflows of various patterns of evacuation.

238

D. Yanagisawa et al.

We also discovered that the pedestrian outflow increases by putting an obstacle in front of the exit from our experiments. Our assumption that the pedestrian outflow increases since the obstacle decreases the conflicts at the exit by blocking the pedestrians’ movement is verified by theoretical calculation using friction functions. Acknowledgements. This work is supported by the Japan Science and Technology Agency and the Japan Society for the Promotion of Science. The experiments in Sec. 8, was supported by the NHK TV program Science Zero.

References 1. Chowdhury, D., Santen, L., Schadschneider, A.: Phys. Rep. 329, 199 (2000) 2. Helbing, D.: Rev. Mod. Phys. 73, 1067 (2001) 3. Kirchner, A., Klupfel, H., Nishinari, K., Schadschneider, A., Schreckenberg, M.: Physica A 324, 689 (2003) 4. Nishinari, K., Kirchner, A., Namazi, A., Schadschneider, A., Nonmembers: IEICE Transactions on Information and Systems E87-D, 3, 726 (2004) 5. Yanagisawa, D., Nishinari, K.: Phys. Rev. E 76, 061117 (2007) 6. Helbing, D., Molnar, P.: Phys. Rev. E 51, 4282 (1995) 7. Song, W., Xu, X., Wang, B.-H., Ni, S.: Physica A 363, 492 (2006) 8. Seyfried, A., Rupprecht, T., Passon, O., Steffen, B., Klingsch, W., Boltes, M.: arXiv:physics/0702004 (2007) 9. Choh, T., Kurabayshi, D., Cheng, J., Funato, T.: Proceedings of 20th SICE Symposium on Decentralaized Autonomous Systems, vol. 63 (2007) (in Japanese) 10. Yanagisawa, D., Tomoeda, A., Nishinari, K.: Proceedings of the Pedestrian and Evacuation Dynamics 2008 (to be published) (2008) 11. Nishi, R., Miki, H., Tomoeda, A., Nishinari, K.: arXiv:0810.4743 (2008)

Part IV

Swarm Intelligence

Understanding the Potential Impact of Multiple Robots in Odor Source Localization Thomas Lochmatter and Alcherio Martinoli

Abstract. We investigate the performance of three bio-inspired odor source localization algorithms used in non-cooperating multi-robot systems. Our performance metric is the distance overhead of the first robot to reach the source, which is a good measure for the speed of an odor source localization algorithm. Using the performance distribution of single-robot experiments, we calculate an ideal performance for multi-robot teams. We carry out simulations in a realistic robotic simulator and provide quantitative evidence of the differences between ideal and realistic performances of a given algorithm. A closer analysis of the results show that these differences are mainly due to physical interference among robots.

1 Introduction With the advances in robotics and chemicals sensor research in the last decade, odor sniffing robots have become an active research area. Notably the localization of odor sources would allow for very interesting robotic applications, such as search and rescue operations, safety and control operations on airports or industrial plants, and humanitarian demining [21] [5] [17] [8]. Many of these applications are time-critical, i. e. odor sources should be found as fast as possible. Moreover, as the structure of plumes in the air is intermittent in both time and space [22], tracking plumes is a challenging problem. Through real-robot [16] [15] and simulation [14] experiments, we have recently shown that the surge-spiral [6] [7] [2] [4] and the surge-cast [15] algorithms are faster and more reliable than pure casting [11] [10] [23] [13] [12] [1] in laminar wind flow. The experiments were run using a single robot, and the result was Thomas Lochmatter and Alcherio Martinoli ´ Distributed Intelligent Systems and Algorithms Laboratory (DISAL), Ecole Polytechnique F´ed´erale de Lausanne (EPFL), Station 2, 1015 Lausanne, Switzerland e-mail: [email protected],[email protected]

240

T. Lochmatter and A. Martinoli

Fig. 1 Distance overhead of single-robot vs. multi-robot systems. (1) Performance gain due to randomness (mathematically derived in Section 3). (2) Performance loss due to physical interference among robots (simulated with a robotic simulator). (3) Performance gained with collaboration among robots (not discussed in this paper)

insofar surprising, as the casting algorithm got much more attention by the research community up to date. In this paper, we are studying the same algorithms with multiple non-cooperating robots. In particular, we compare the performance difference when moving from a single-robot to a homogeneous multi-robot system with 2 or 5 robots. Our performance metric is the distance overhead (traveled distance dt divided by upwind distance du ), which is an excellent indicator for the speed of a plume following algorithm on a holonomic robot [15]. Moreover, we only require one robot to reach the odor source, and use the distance overhead of the first robot to reach the source as the performance of the robotic team. As sketched in Figure 1, the difference in distance overhead between single-robot (A) and non-cooperating multi-robot systems (C) consists of two components. First, randomness due to the noise in the system boosts the performance (Figure 1 (1)). This performance gain can be calculated by using the distribution of the distance overhead of single-robot experiments, and would be achieved if the robots were not interacting with each other (B). Second, physical interference among the robots result in a loss in performance (Figure 1 (2)), which we quantify by running simulations in a realistic robotic simulator [19]. Cooperation among robots (D, not discussed in this paper) would again result in a performance gain. Multi-robot odor source localization experiments with an algorithm called spiral surge (which is close to the surge-spiral algorithm used here) have previously been carried out by Hayes et al. [6] [7]. Hayes ran experiments with up to 6 real robots, and up to 10 robots in simulation. Results showed that increasing the number of robots is beneficial in terms of time to find the source. To our knowledge, casting strategies have never been tested with multiple robots. In two other projects [9] [18], multi-robot odor source localization algorithms based on Particle Swarm Optimization (PSO) were tested in simulation. In both papers, the robots were communicating with each other. The remainder of this paper is structured as follows. In Section 2, we present the three algorithms used in this paper. In Section 3, we derive the ideal performance

Understanding the Potential Impact of Multiple Robots

241

for non-cooperative multi-robot systems. The simulator and the odor propagation model are introduced in Section 4. Finally, we discuss the results in Section 5 and conclude in Section 6.

2 Algorithms All three algorithms used in this paper are bio-inspired [20] and use binary odor information. That is, they either perceive the odor or do not perceive any odor, but ignore different concentrations levels. Finally, all three algorithms need a wind sensor to measure the wind direction. With the casting algorithm, the robots move in a zig-zag pattern under an upwind angle β towards the source. With surge-spiral, the robots move upwind as long as they are in the plume, and spiral with a gap length dgap to reacquire the plume whenever they lose it. The surge-cast algorithm, finally, works in a very similar fashion, except that the robot casts in crosswind direction for a distance dcast to reacquire the plume. A detailed description of these algorithms can be found in [15]. To avoid collisions, all robots are running a Braitenberg obstacle avoidance algorithm using 9 on-board infrared proximity sensors. Both algorithms run (independently from each other) in parallel at all times. While obstacle avoidance has virtually no influence on the behavior of the robot in open space, it overrides the plume tracking algorithm when the robot is close to an obstacle (i. e., another robot in our case). Finally, a robot gives up and stops after having lost the plume for too long, or reached the arena boundary. In this paper, we only consider plume traversal and intentionally omit plume finding (i. e. randomized or systematic search until the plume is found) and source declaration (i. e. declaring that the source is in close vicinity), to prevent those two parts from interfering in the results. Hence, the robots start in the plume, and source declaration is done by a supervisor (ideal source declaration). Experiments are considered successful as soon as the first robot has come in physical vicinity of the source, and unsuccessful if all robots gave up.

3 Expected Performance of Multi-robot Experiments Assume a performance value q that can be associated with each experimental run. In this paper, this metric is the distance overhead (traveled distance dt divided by upwind distance du ) of the first robot that reaches the source. Hence, a small q value stands for a good performance, with q = 1 being the optimum. The distance overhead of an experiment with a single-robot algorithm (with a fixed set of parameters) can be expressed as a distribution Q1 , which can be approximated by performing a large number of runs. Examples of such distributions estimated with 200 runs are shown in Figure 2.

242

T. Lochmatter and A. Martinoli

50

Q2 (2 robots)

40

Q5 (5 robots) 30

20

10

0 1.22

1.24

1.26

1.28

1.3

1.32

Q2 (2 robots)

40

Q5 (5 robots) 30

20

10

0

1.34

100

Q1 (1 robot)

1

1.1

Distance overhead

casting, β

Estimated probability density

Q1 (1 robot)

Estimated probability density

Estimated probability density

50

1.2

1.3

1.4

Q1 (1 robot) Q2 (2 robots)

80

Q5 (5 robots) 60

40

20

0

1

Distance overhead

= 25o

surge-spiral, dgap =22 cm

1.05

1.1

1.15

1.2

Distance overhead

surge-cast, dcast =34 cm

Fig. 2 Q1 : Experimentally measured distribution of the distance overhead with a single robot. Q2 and Q5 : The expected distributions for 2 resp. 5 robots, based on the assumption that the robots do not physically interfere with each other. The triangles on top of the diagram indicate the mean values of the respective distributions

If two independent robots are going for the same source, their performances qa and qb are random samples drawn from Q1 . Clearly, the smaller of these two numbers (corresponding to the faster robot) will set the overall performance of the team, qab = min(qa , qb )

(1)

Hence, the performance distribution of a system with two independent robots is the distribution of qab , and can be expressed as Q2 (q) =

1 cQ



[min(qa , qb ) = q] Q1 (qa )Q1 (qb ) dqadqb

with cQ =



Q1 (qa )Q1 (qb ) dqa dqb

(2) (3)

where [· ] stands for the Iverson bracket. Generalizing this for N robots is straightforward.

3.1 Calculating QN Closed-form expressions for QN only exist for a few well-known distributions. If Q1 is exponentially distributed with mean λ1 , for instance, then QN is exponentially distributed with mean N1λ . The algorithms used in this paper yield complicated distributions, however, and an approximation by an exponential distribution would be very rough for the surgespiral and surge-cast algorithms, and not justifiable for the casting algorithm. We therefore calculated the distributions for multiple robots numerically, by randomly sampling from the distribution Q1 (Monte-Carlo simulation). Formally, we estimated the distribution QN with 100000 samples of the form min(q1 , q2 , ..., qN )

(4)

Understanding the Potential Impact of Multiple Robots

243

where q1 , q2 , ..., qN are randomly selected performance samples of the single-robot runs. Distributions obtained in this way for 2 and 5 robots executing the casting algorithm are shown in Figure 2. On that figure, it can be observed how the distribution and its mean value shift towards the left (lower distance overhead) as the number of robots increases.

4 Simulation Experiments We are using Webots [19] for the experiments. Webots is a commercial realistic robotic simulator, which ships with a calibrated model of the Khepera III robot that we used for the real-robot experiments [15]. The simulation environment (Figure 4) was enhanced with a wind and odor propagation model, and the robot model was extended with the corresponding sensors to measure the odor concentration and a wind direction (Figure 3). The simulation setup was kept close to the setup in the wind tunnel that we used in previous work for the experiments with real robots [16] [15]. The simulation time step, ∆ t, was set to 32 ms.

4.1 Experimental Arena The experimental arena was a rectangular area of 18 m length and 4 m width, which corresponds roughly to the dimensions of the wind tunnel. At 1 m from one end of the arena, a circular odor source of radius 12 cm was placed. The robots were placed at roughly 14.5 m downwind from the source.

4.2 Advection Model A constant wind field of 1 m/s was used, which corresponds to a constant laminar flow comparable to the one in the wind tunnel. In the coordinate system indicated in Figure 4, the wind vector at position u, a(u), can be written as ⎛ ⎞ 1 (5) a(u) = ⎝ 0 ⎠ 0

4.3 Odor Propagation Model The odor propagation model closely resembles the filament-based model proposed by Farrell et al. [3]. This model is easy to implement and requires only a very limited amount of CPU power. Yet, it generates an intermittent plume which is similar to the real plume in the wind tunnel.

244

T. Lochmatter and A. Martinoli

Odor is thereby simulated as a set of filaments (i = 0, ..., N), each containing a constant amount s = 8.3 · 109 of molecules or particles. Each filament is defined by its position, pi,t , and its width, wi,t . In each time step, the position of a filament is updated according to the wind flow and a stochastic process: pi,t+∆ t = pi,t + a(pi,t )∆ t + v p

(6)

The stochastic component v p is a vector of three independent Gaussian random variables, N(0, σ p2 ), with standard deviation σ p = 0.1 m. To model molecular dispersion, filaments become wider with time while their peak concentration decreases. The width of a filament evolves as wi,t+∆ t = wi,t +

γ 2wi,t

with γ = 4 · 10− 7

(7)

Our virtual odor source released 100 such filaments per second with an initial width of wi,0 = 10 cm and an initial position which was uniformly distributed over the circular area of the source. This resulted in a plume comparable to the real plume in the wind tunnel.

4.4 Odor Sensor Model The odor concentration at time t and position u was calculated as the sum over the concentration contribution of all filaments, N

Ct (u) = ∑ ci,t (u)

(8)

i=0

and each filament i contributed s ci,t (u) = 3 exp wi,t



| u − pi,t | w2i,t



(9)

to the concentration. Hence, the concentration decayed exponentially with increasing distance from the center of a filament. The virtual odor sensor reported this concentration Ct (u) without adding any additional noise, as the perceptual noise related to the chemical-to-electrical transduction is negligible even on the real platform [15]. Furthermore, since the concentration is anyway thresholded and filtered through dlost by the algorithms in use here, a precise calibration of the odor propagation and odor sensor model was not required.

4.5 Wind Direction Sensor Model The wind sensor reported a noisy wind measurement,

Understanding the Potential Impact of Multiple Robots

245

Fig. 3 Simulated Khepera III robot equipped with an odor sensor (small cylinder on top of the robot) and a wind sensor (big cylinder). The hexagons in the air represent odor filaments

Fig. 4 Simulated environment (18 m by 4 m arena) in Webots

as (u) = a(u) + va

(10)

where va was a vector with samples of a zero-mean normal distribution (N(0, σa2 )). Since the wind field was constant in all our simulations, the reported value in world coordinates was simply ⎛ ⎞ ⎛ ⎞ N(0, σa2 ) 1 as (u) = ⎝ 0 ⎠ + ⎝ N(0, σa2 ) ⎠ (11) with σa = 0.1 m 0 N(0, σa2 ) This vector was rotated into the local reference system of the robot to account for the robot’s pose.

4.6 Experiments For all three algorithms, we run experiments with 9 different parameters (upwind angle β for casting, spiral gap dgap for surge-spiral, and cast distance dcast for surgecast), each with 1, 2, or 5 robots. For β = 25o , dgap = 22 cm and dcast = 34 cm, we performed 200 independent runs and calculated the ideal performance as described in Section 3. For all other configurations, 50 runs were carried out. In each run, the robots were released in the odor at fixed positions (evenly spaced) between 14.5 m and 16 m downwind from the source. If one robot reached the odor source, the run was stopped and considered successful. During the run, the trajectory, the measured odor concentration and the measured wind direction were recorded for each simulation step. Distance and upwind distance were derived from the trajectory.

246

T. Lochmatter and A. Martinoli

Casting: β=25°

1.25

u

Traveled distance d / upwind distance d [m/m]

1.3

1.2

t

1.15

1.1

1.05

1

1

2

5

Robots

(a)

(b)

Fig. 5 (a) Results obtained with the casting algorithm. The error bars indicate the 95 % confidence interval for the mean. (b) Close-up for β = 25o . The thin arrows indicate the intrinsic performance gain by passing from a single-robot to a multi-robot system (Figure 1 (1)), while the thick arrows indicate the performance loss due to physical interaction between the robots (Figure 1 (2))

The forward speed of the robot (on straight lines) was 10.6 cm/s and therefore same as with the real-robot experiments in the wind tunnel. The plume threshold was set to c = 100.

5 Results and Discussion 5.1 Casting The results for the casting algorithm are presented in Figure 5. The differences between the single-robot and the multi-robot experiments are very small and statistically not significant for most configurations. However, as a general trend, multiple robots seem to yield slightly better performance for upwind angles β > 20o , and worse performance otherwise. As Figure 5 (b) reveals, even the ideal performance (for robots that are not physically interfering) is not much better than the single-robot performance. Indeed, single-robot experiments with the casting algorithm yield a compact — almost normal — performance distribution with a small variance, and the resulting ”left shift“ of the distribution for multiple robots is small. A noticeable gain can be observed for the success rate, however. For small angles, where the success rate with a single robot is small, a team of robots can achieve very high success rates. This robustness is an advantage often cited in the context of multi-robot systems. Surprisingly, physical interference seems not to have a big

Understanding the Potential Impact of Multiple Robots

247 Surge−spiral: d

=22 cm

gap

1.25

u

Traveled distance d / upwind distance d [m/m]

1.3

1.2

t

1.15

1.1

1.05

1

1

2

5

Robots

(a)

(b)

Fig. 6 (a) Results obtained with the surge-spiral algorithm. (b) Close-up for dgap = 22 cm

influence here. As the following table shows, the actual success rates obtained in the multi-robot experiments are close to the expected success rates calculated based on the success rate of the single-robot runs. 1 robot 2 robots 5 robots actual actual expected actual expected Casting, β = 5o 0.66 0.76 0.884 1.0 0.995 o Casting, β = 10 0.78 0.96 0.952 1.0 0.999

5.2 Surge-Spiral The picture for the surge-spiral algorithm looks pretty different. As the performance distribution of single-robot runs resembles an exponential distribution, its mean value decreases as N1 with increasing numbers N of robots. Hence, large performance gains are expected. For small spiral gaps, no performance gain is noticed in the simulation results. For large gaps, however, the difference between the single-robot and the multi-robot runs are significant. Even though the theoretical model would expect an even larger difference, the multi-robot runs were clearly faster than the single-robot runs and achieved similarly good results as the single-robot runs with small spiral gaps. The reason for this is that a spiraling robot spends enough time aside the plume, such that other robots can safely overtake. This could be interpreted as an indirect and implicit coordination scheme (without communication), whereby robots losing the plume try to make room for other robots in the plume.

248

T. Lochmatter and A. Martinoli Surge−cast: d

=34 cm

cast

1.25

u

Traveled distance d / upwind distance d [m/m]

1.3

1.2

t

1.15

1.1

1.05

1

1

2

5

Robots

(a)

(b)

Fig. 7 (a) Results obtained with the surge-cast algorithm. (b) Close-up for dcast = 34 cm

5.3 Surge-Cast The results for the surge-cast are similar: the longer the cast distance, the more performance is gained by using multiple robot. This has to be taken with a grain of salt, though, since longer cast distances yield worse performance in the single-robot case and are therefore not desired anyway. Hence, in well-configured systems with near-optimal cast distances (here 27 cm - 34 cm), no performance gains are visible. Contrary to the casting experiments, using multiple robots does not increase the robustness of the algorithm here. In some cases, the success rate even got worse. The surge-cast algorithm in its present form is clearly not robust with respect to physical collisions. Especially during plume reacquisition, a robot blocking the way at the plume boundary can cause another robot to lose the plume completely. This could certainly be improved by adaptively increasing the cast distance until the plume is found.

6 Conclusion We carried out single-robot and non-cooperative multi-robot odor source localization experiments in simulation with three different bio-inspired algorithms, and compared their results in terms of success rate and distance overhead. The setup was similar to the single-robot experiments carried out with real Khepera III robots in the wind tunnel [15]. While the theoretically ideal performance of the multi-robot teams was expected to be significantly better than that of the single-robot runs, the actual performance was found to be comparable for most configurations. Statistically significant differences were mainly found for the surge-spiral algorithm with large spiral gaps.

Understanding the Potential Impact of Multiple Robots

249

With such large gaps, the robots leave the plume for a significant time and distance, which allows other robots to overtake without interference. This underlines the negative effect of physical interference among the robots on the team performance. In particular, uncoordinated teams of robots have troubles overtaking each other during plume following. A simple, local coordination scheme dealing with this problem would presumably offer a significant performance gain. In future work, we will test the algorithms in turbulent flow and/or meandering plume conditions, and test different multi-robot coordination schemes in simulation and using real robots. Acknowledgements. This work was supported by the National Competence Center in Research on Mobile Information and Communication Systems NCCR-MICS, a center supported by the Swiss NSF under grant number 5005-67322.

References 1. Balkovsky, E., Shraiman, B.I.: Olfactory search at high reynolds number. PNAS 99(20), 12589–12593 (2002) 2. Berlanger, J.H., Willis, M.A.: Adaptive control of odor-guided locomotion: behavioral flexibility as an antidote to environmental unpredictability. Adaptive Behavior 4(3-4), 217–253 (1996) 3. Farrell, J.A., Murlis, J., Long, X., Li, W., Card´e, R.T.: Filament-based atmospheric dispersion model to achieve short time-scale structure of odor plumes. Environmental Fluid Mechanics 2, 143–169 (2002) 4. Ferri, G., Caselli, E., Mattoli, V., Mondini, A., Mazzolai, B., Dario, P.: A biologicallyinspired algorithm implemented on a new highly flexible multi-agent platform for gas source localization. In: Proceedings of the First IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics (BIOROB 2006) (February 2006) 5. Gage, D.W.: Many-robot MCM search systems. In: Proceedings of the Autonomous Vehicles in Mine Countermeasures Symposium, vol. 9, pp. 56–64 (Appril 1995) 6. Hayes, A.T., Martinoli, A., Goodman, R.M.: Distributed odor source localization. IEEE Sensors Journal 2(3), 260–271 (2002) 7. Hayes, A.T., Martinoli, A., Goodman, R.M.: Swarm robotic odor localization: Off-line optimization and validation with real robots. Robotica 21, 427–441 (2003) 8. Ishida, H., Nakamoto, T., Moriizumi, T., Kikas, T., Janata, J.: Plume-tracking robots: A new application of chemical sensors. Biological Bulletin (200), 222–226 (2001) 9. Jatmiko, W., Sekiyama, K., Fukuda, T.: A pso-based mobile robot for odor source localization in dynamic advection-diffusion with obstacles environment. IEEE Computational Intelligence Magazine, 37–51 (May 2007) 10. Kuenen, L.P.S., Rowe, H.C.: Cowpea weevil flights to a point source of female sex pheromone: analyses of flight tracks at three wind speeds. Physiological Entomology 31(2), 103 (2006) 11. Li, W., Farrell, J.A., Card´e, R.T.: Tracking of fluid-advected odor plumes: Strategies inspired by insect orientation to pheromone. Adaptive Behavior 9(3-4), 143–170 (2001) 12. Li, W., Farrell, J.A., Pang, S., Arrieta, R.M.: Moth-inspired chemical plume tracing on an autonomous underwater vehicle. IEEE Transactions on Robotics 22(2), 292–307 (2006)

250

T. Lochmatter and A. Martinoli

13. Lilienthal, A.J., Reiman, D., Zell, A.: Gas source tracing with a mobile robot using an adapted moth strategy. In: Autonome Mobile Systeme (AMS), Fachgespr¨ach, GDI, vol. 18, pp. 150–160 (December 2003) 14. Lochmatter, T., Martinoli, A.: Simulation experiments with bio-inspired algorithms for odor source localization in laminar wind flow. In: Proceedings of the The Seventh International Conference on Machine Learning and Applications (ICMLA 2008). IEEE, Los Alamitos (2008) 15. Lochmatter, T., Martinoli, A.: Tracking odor plumes in a laminar wind field with bioinspired algorithms. In: Proceedings of the 11th International Symposium on Experimental Robotics 2008 (ISER 2008). Springer Tracts in Advanced Robotics (2010), Athens, Greece (July 2008) (to appear) 16. Lochmatter, T., Raemy, X., Matthey, L., Indra, S., Martinoli, A.: A comparison of casting and spiraling algorithms for odor source localization in laminar flow. In: Proceedings of the 2008 IEEE International Conference on Robotics and Automation (ICRA 2008), pp. 1138–1143 (May 2008) 17. Long, M., Gage, A., Murphy, R., Valavanis, K.: Application of the distributed field robot architecture to a simulated demining task. In: Proceedings of the 2005 IEEE International Conference on Robotics and Automation (ICRA 2005), pp. 3193–3200 (April 2005) 18. Marques, L., Nunes, U., de Almeida, A.T.: Particle swarm-based olfactory guided search. Autonomous Robots 20(3), 277–287 (2006) 19. Michel, O.: Webots: Professional mobile robot simulation. International Journal of Advanced Robotic Systems 1(1), 39–42 (2004) 20. Russell, R.A.: Odour Detection by Mobile Robots. World Scientific Series in Robotics and Intelligent Systems, vol. 22. World Scientific Publishing Company, Singapore (1999) 21. Settles, G.S.: Sniffers: Fluid-dynamic sampling for olfactory trace detection in nature and homeland security—the 2004 freeman scholar lecture. Journal of Fluids Engineering. Transactions of the ASME 127, 189–218 (2005) 22. Vergassola, M., Villermaux, E., Shraiman, B.I.: ‘Infotaxis’ as a strategy for searching without gradients. Nature 445, 406–409 (2007) 23. Webb, B., Harrison, R.R., Willis, M.A.: Sensorimotor control of navigation in arthropod and artificial systems. Arthropod Structure and Development 33, 301–329 (2004)

Analyzing Multi-agent Activity Logs Using Process Mining Techniques A. Rozinat, S. Zickler, M. Veloso, W.M.P. van der Aalst, and C. McMillen

Abstract. Distributed autonomous robotic systems exhibit complex behavior that—although programmed, but due to the impact of the environment— only materializes as the process unfolds. Thus, the actual behavior of such a system cannot be known in advance but must be observed to be evaluated or verified. In this paper we propose to use process mining techniques to extract, compare, and enhance models of the actual behavior of a multi-agent robotic system through analyzing collected log data. We use the example of robot soccer as such a multi-agent robotic system, and we demonstrate which types of analysis are currently possible in the context of the process mining tool set ProM.

1 Introduction Robotic systems are growing more and more complex as they seek, for example, to be self-reconfiguring, self-organizing, or working in teams. While their behavioral logic is of course programmed, and, thus, in principle predictable, the more autonomous a robot grows, or the more it adapts to its environment, the more it is true that the actual behavior of the system cannot really be known in advance anymore. For example, in a team of robots, the overall system behavior is determined through interaction among multiple robots. Or, A. Rozinat and W.M.P. van der Aalst Information Systems Group, Eindhoven University of Technology, NL-5600 MB, Eindhoven, The Netherlands e-mail: {a.rozinat,w.m.p.v.d.aalst}@tue.nl A. Rozinat, S. Zickler, M. Veloso, and C. McMillen Computer Science Department, Carnegie Mellon University, Pittsburgh PA 15213-3890, USA e-mail: {szickler,veloso,mcmillen}@cs.cmu.edu

252

A. Rozinat et al.

if robots interact with humans, their actions are influenced by the actions of the human. Thus, the question whether this overall behavior corresponds to the intended system behavior can only be answered through observation. Process mining techniques use log data to analyze observed processes and have been successfully applied to real-life logs from, e.g., hospitals, banks, municipalities etc. (see [2] for one of many real-life applications). The basic idea of process mining is to discover, monitor and improve real processes (i.e., not assumed processes) by extracting knowledge from event logs. Today many of the activities occurring in processes are either supported or monitored by information systems. However, process mining is not limited to information systems and can also be used to monitor other operational processes or systems, such as complex X-ray machines, high-end copiers, or web services. The common denominator in the various applications of process mining is that there is a notion of a process and that the occurrences of activities are recorded in so-called event logs [1]. In this paper, we use log data collected by the CMDragons team during the international robot soccer competition ’RoboCup’ 2007 to investigate the applicability of process mining techniques to a multi-agent robotic system. In the context of robot soccer, the motivation for analyzing log data is two-fold: • Self-analysis: While detailed logs are recorded during the competitions, the evaluation of a game is carried out mostly through watching the accompanying video recordings. A wealth of detailed data are available, but usually they are not analyzed in a structured way. We argue that a systematic and more high-level analysis of these log data can help to obtain an overall picture, and to, for example, discover transient faulty states that—due to being held only for a short time—are difficult to be observed by the human eye. • Opponent analysis: Obtaining behavioral models of the opponent robots is even more interesting as their intentions are hidden and the only way to derive knowledge about their strategic behavior is through observation. We envision the application of process mining techniques in combination with activity recognition [9, 7], which is able to identify high-level actions (for example that a robot is “attacking”) based on low-level behaviors (such as the robot’s position and velocity). Without loss of generality, we can restrict ourselves to self-analysis, since the described techniques can be equally applied for the purpose of opponent analysis given that appropriate activity recognition mechanisms are in place. The paper is organized as follows. First, we describe the domain of robot soccer that is used in the remainder of this paper as an example of a multirobot system (Section 2). Then, we introduce process mining and explain how process mining can be applied in the context of this example domain (Section 3). Next, the log data that are used as input for the process mining techniques are described (Section 4) and some illustrative analysis results are presented (Section 5). Section 6 concludes the paper.

Analyzing Multi-agent Activity Logs Using Process Mining Techniques

253

2 Robot Soccer: A Multi-agent System Behavioral multi-robot systems are control architectures where multiple agents coordinate the execution of different individual tactical approaches, called behaviors, typically in order to achieve a common goal. One particularly interesting behavioral multi-agent domain, which we are going to use as the data source for our experiments, is the RoboCup Small Size League. Here, the objective is to have two teams of small robots compete in a game of miniature soccer, without human intervention. In the Small-Size league, each team normally consists of five homogeneous, omni-directional robots which are remotely controlled by an off-board computer. The computer obtains its observations from two overhead cameras mounted above the soccer field which are then processed to provide very accurate estimates of positions and orientations of all the players on the field. Challenges of this domain include its very fast pace, and its complex tactical properties. A scene from a typical robocup game is shown in Figure 1.

Fig. 1 A scene of a RoboCup Small-Size League game

This paper utilizes real game data collected by Carnegie Mellon University’s Small-Size team “CMDragons” [5]. The software architecture of the team’s offboard control system is shown in Figure 2. The server component in this diagram performs computer vision and manages communication with the robots. The intelligence of the system arises from the soccer component which embodies all autonomous decision making processes. The heart of the soccer component is its behavioral control architecture, known as “Skills, Tactics, and Plays” (STP) [4]. Within STP, multi-agent coordination and team-based strategic decisions are performed by chosing from a collection of “plays”. A play is defined as a list of role assignments for multiple robots. For example, in robot soccer, one might imagine an “offensive” play, which assigns most robots to be attackers, and some other robots to be defenders. The applicability of a particular play depends on whether that play’s pre-conditions are currently met, given the value of some variables of the domain. If multiple plays are simultanously applicable then one of them is chosen probabilistically. The role assignment (e.g. which particular robots become an attacker) is then performed dynamically based on the robots’ positions and availability. Individual robot roles are called tactics and are typically implemented as state-machines consisting of lower level navigational functions, called skills. Examples of soccer tactics include “Attacker”, “Defender”, and “Goalie”.

254

A. Rozinat et al.

Fig. 2 The general architecture of the CMDragons offboard control software

The data used in our experiments originate from the performance of the CMDragons team at RoboCup 2007. The data was gathered over four round robin games against teams named “Wright Eagle”, “RoboDragons”, “BSmart”, and “Botnia”, followed by the quarter-final, semi-final, and final games against “Eagle Knights”, “ZJUNlict”, and “Plasma-Z”, respectively. For each game, the system has generated a log file describing the game observations, as well as the CMDragon’s internal system state. Log entries carry the currently executed play and tactic names of each robot and additional globally observed variables, such as positions, and velocities of the ball and robots. Furthermore, they include the status of so-called referee events which indicate the change of the global game status such as, e.g., a switch from a “game-stoppage” to a “kickoff”. Log entries are recorded at 60Hz in unison with the system’s main processing loop, which leads to a new log entry about every 17 milliseconds.

3 Process Mining Process mining is a field of analysis techniques that focuses on mining behavioral aspects from log data. Since the mid-nineties several groups have been concentrating on the discovery of process models from event-based data. Process models are structures—usually directed graphs—that model behavior. In [3] an overview is given of the early work in this domain. Despite the many relations to some work discussed in the Machine Learning domain, there are also many differences as the targeted process models reside at the net level (e.g., Petri nets) rather than sequential or lower level representations (e.g., Markov chains, finite state machines, or regular expressions). Therefore, process mining needs to deal with various forms of concurrency. Over the years, the notion of process mining has broadened to the general idea of extracting non-trivial and useful information from process execution logs [1]. For example, in the context of the robot soccer domain described in the previous section, the real-world process of interest is a robot soccer match (see Figure 3). Multiple robots interact with each other in the process of the game, and we are able to record information about their activities. These so-called “event logs” are the starting point for process mining techniques, and they can be used to derive and enrich higher-level models, or to compare

Analyzing Multi-agent Activity Logs Using Process Mining Techniques Fig. 3 Process mining in the context of robot soccer

Robot Soccer

255

Robot

models / analyzes

records events

discovery

Process model

conformance

Event logs

extension

them to potentially existing models of behavior. Generally, we distinguish three classes of analysis: Discovery. Extracting a new model from the event log. In the context of robot soccer, for example, we may want to see process models of the overall team behavior, or of individual robot behavior. Conformance. Comparing an existing model to the behavior observed in the log. In the robot soccer example one could, for instance, compare the behavior of the opposing team with the own behavioral models or compare the observed behavior with some normative or descriptive model. Extension. Projecting additional information onto an existing model. Note that many aspects of a process (such as timing) cannot be known beforehand as they depend on the environment (for example, the opposing team). Hence, existing models can be enriched with derived information. ProM [1] is a tool set that supports a wide variety of process mining techniques1 . Furthermore, due to its plug-able architecture, it enables researchers to quickly implement new algorithms without spending any efforts on common tasks such as loading logs, displaying discovered models etc. In this study, we did not develop any custom analysis plug-in but used functionality that is readily available in ProM. Some sample analysis results are described in more detail in Section 5. However, in the next section we first take a closer look at the activity logs that form the input data for the process mining techniques.

4 Activity Logs As described in Section 2, there are different levels of robot behavior. The high-level actions that we want to analyze are the tactics, which are the different roles that a robot can have, such as “mark” or “position for pass”. Since ProM uses the M ining XML (MXML) format to read event logs, the 1

The software including source code and documentation can be freely downloaded from prom.sf.net (see also our website processmining.org).

256

A. Rozinat et al.

data collected by the CMDragons team needed to be converted into this format. The basic structure of MXML is very simple: a process log consists of a set of process instances, which in turn each contain a sequence of events. A process instance, also referred to as case, trace, or audit trail, is one particular realization of the process, while events correspond to concrete steps that are undertaken in the context of that process for the particular process instance. Furthermore, each event carries a timestamp and many contain additional data. Note that already during log conversion different views on the process can be taken, since the definition of what is to be considered a process instance determines the scope of the process to be analyzed. For example, in the context of a robot soccer match one could be interested in the overall team formations over multiple games, as well as individual robot behaviors within a single game. Thus, different conversions can be leveraged to obtain different views on the same process. Using a custom plug-in of the ProMimport framework [6] we have created both individual robot logs and team logs. Because the overall team behavior is globally orchestrated, we can easily get information about the current role of all the robots in the team at each point in time, and we consider a team formation by abstracting from which robot engages in which role. Thus, in a team log we see an action as a set of roles. Furthermore, we abstract from the goalie (as there is always one goalie). An excerpt of an event in such a team log is shown in the following MXML log fragment. The event (AuditTrailEntry) was recorded in the context of the game against “Plasma-Z” (ProcessInstance) on 8 July 2007 at 12:57:18 according to US Eastern Time Zone (Timestamp), and refers to the team formation “Mark Mark Wall Wall” (WorkflowModelElement) while the ball was in possession of the CMDragons (Data) etc. The dots indicate that the log contains further process instances, and the process instance further events. Furthermore, the event contains more data attributes than is shown here. ... ...

1 ...

Mark Mark Wall Wall complete 2007-07-08T18:57:18.709+02:00 ... ...

The complete team log contains more than half a million (624,801) events that are distributed over 7 process instances. Because we are mainly interested in changes of behavior, we subsequently filter out repetitions of the same team formation. This way, we yield a much more condensed log focusing only on state changes, while preserving information about the timing

Analyzing Multi-agent Activity Logs Using Process Mining Techniques

257

Start 1

1 DefendCircle Wall Wall Wall 158

8 DefendCircle Position Position Position 8

15

9

80

DefendCircle Mark Mark Wall 9

8

9

DefendCircle PositionForPass PositionForPass PositionForReady 8

15

20

7

17

32

46

72

20

21

32

Mark Mark Wall Wall 20

DefendCircle PositionForPass SetPlayKick Wall 22

Attacker DefendCircle DefendCircle Mark 175

DefendCircle PositionForPass PositionForPass SetPlayKick 22

15

Mark Mark Position Wall 15

22

23

PositionForPass PositionForPass SetPlayKick Wall 23

11

Attacker DefendCircle PositionForPass PositionForPass 79

6

1 End 1

9 DefendCircle DefendCircle Mark Wall 32

32

41

22

5

DefendCircle DefendCircle PositionForPass SetPlayKick 5

82

32

12

DefendCircle Position ReceiveChip SetPlayKick 9

5

9

40 Attacker DefendCircle DefendCircle PositionForPass 177

23

Fig. 4 Process model discovered by the Heuristics Miner based on the team log of the final game between CMDragons against “Plasma-Z”. Each rectangle corresponds to a team formation of the CMDragons in that game (without the goalie)

behavior through the time stamps. The filtered team log now contains only 2,183 events for all the 7 games, with the shortest process instance listing 63 and the longest 762 team formation changes. The same can be done for individual robot logs.

5 Analysis and Results Due to space limits we can not go into detail about individual mining techniques, and only consider two illustrative examples of discovery (cf. Section 3), namely (a) process model discovery and (b) decision rule mining. (a) Based on the team log, we use the Heuristics Miner [10] to automatically construct a process model of the team formations in the final game against “Plasma-Z”, which is depicted in Figure 4. It shows the causal dependencies between activities and provides an overview about the actual flow of the process, whereas each rectangle corresponds to a team formation of the CMDragons in that game (the numbers reflect the frequencies at which the team formation changes were observed). Now consider Figure 5, which was created from an individual robot log, and thus depicts the discovered process model of one individual robot in the same game. We can see that the robot was frequently changing into the role “Wall” (130 times) and that it was, for example, frequently switching from the role “Wall” into “PositionForPass” (67 times) and back (73 times). Note that these models could now be further

258

A. Rozinat et al. Start 1

1 DefendCircle 106

26

27

48 Attacker 86

14

38

24

1 End 1

31

30

66

45

60 52

20

45

39

Mark 91

21

39

46

27

43

PositionForPass 123

73

12

31

5

Position 15

1

5

43

67 Wall 130

18

54

65

ReceiveChip 5

15

PositionForReady 1

1 SetPlayKick 18

Fig. 5 Process model of one individual robot’s behavior

enhanced by making use of the time information in the log and visualizing the length of stay in each of these states by different colors. (b) We apply the Decision Miner [8] in combination with the Weka machine learning workbench [11] to discover under which conditions which team formation is being chosen by the system. For this, we create a Petri net process model that represents the playbook of the soccer component (cf. Figure 2), which is partly shown in Figure 6, and we restrict the used data attributes to those used in the pre-conditions of the world model. As described earlier, the internal behavioral model assigns probabilities to the different plays, which are active as soon as their pre-conditions are fulfilled. Apart from these pre-conditions, every team formation is possible at each point in time (represented by “playing game” in Figure 6). Note that the internal model is more general than the discovered models in Figure 4 and Figure 5, which depict typical action flows more closely. Thus, the discovered models can provide real insight into the process as it took place, different games can be compared etc. It is interesting that we can discover the same or similar rules to a large extent. For example, the discovered rule ‘their kickoff => Mark Mark Wall Wall ’ coincides with the original one and has both precision and recall of 1.0. The discovered rule ‘their freekick AND !their side => DefendCircle Mark Wall Wall ’ is slightly more general as the original rule was ‘their freekick AND our side AND

Analyzing Multi-agent Activity Logs Using Process Mining Techniques

259

their_kickoff Mark Mark Wall Wall

before game

Start

playing game

their_freekick AND !their_side DefendCircle Mark Wall Wall

...

End

after game

Fig. 6 Petri net process model reflecting the world model, where each of the team formations is possible at every point in time, if certain pre-conditions are met. The discovered rules (two examples are shown) largely coincide with the internal rules

!our corner ’.

The overall accuracy of the discovered rules (using 10-fold cross validation over the whole team log) is between 63 and 74 % (depending on the classification algorithm). If there are mismatches, then this is mainly because some plays share the same pre-conditions and are then selected randomly. If we include additional attributes not directly used in the pre-conditions (e.g., referee status) the accuracy exceeds even 90 %. While it is less useful to rediscover one’s own rules, this seems directly useful in the context of analyzing the opponent’s behavior (“When do they react how?”).

6 Conclusion In this paper, we have shown the potential of structured log analysis to gain more high-level insight into the behavior of multi-robot systems. We have demonstrated based on the robot soccer domain that it is possible to apply process mining to multi-agent activity logs. Many other techniques are applicable but have not been mentioned. For example, while the global observation system is very accurate and provides high-qualitative data, in turbulent phases of the game sometimes not all attributes can be reliably determined. Thus, it can happen that the ball possession is “ours” and “theirs” at the same time. Using linear temporal logic (LTL) one can check such pre-defined properties (‘(ball possession ours AND ball possession theirs )’) and further investigate those cases to improve the situation. Finally, custom implementations in the context of the ProM framework are possible. They have the advantage that they—while building on common functionality—enable domain-aware and specialized solutions. Acknowledgements. This research is supported by the IOP program of the Dutch Ministry of Economic Affairs. The authors also thank the CMDragons team for sharing their RoboCup log data, and Douglas Vail who helped us to create the

260

A. Rozinat et al.

ASCII logs used for conversion to the MXML format. Furthermore, we thank all ProM developers for their on-going work on process mining techniques. Special thanks to Christian W. G¨ unther for NikeFS2, which makes it possible to work with large real-world logs, and for his GUI redesign, which continues to impress people and makes using ProM so much more fun.

References 1. van der Aalst, W.M.P., van Dongen, B.F., G¨ unther, C.W., Mans, R.S., de Medeiros, A.K.A., Rozinat, A., Rubin, V., Song, M., Verbeek, H.M.W., Weijters, A.J.M.M.: ProM 4.0: Comprehensive Support for Real Process Analysis. In: Kleijn, J., Yakovlev, A. (eds.) ICATPN 2007. LNCS, vol. 4546, pp. 484–494. Springer, Heidelberg (2007) 2. van der Aalst, W.M.P., Reijers, H.A., Weijters, A.J.M.M., van Dongen, B.F., Alves de Medeiros, A.K., Song, M., Verbeek, H.M.W.: Business Process Mining: An Industrial Application. Information Systems 32(5), 713–732 (2007) 3. van der Aalst, W.M.P., van Dongen, B.F., Herbst, J., Maruster, L., Schimm, G., Weijters, A.J.M.M.: Workflow Mining: A Survey of Issues and Approaches. Data and Knowledge Engineering 47(2), 237–267 (2003) 4. Browning, B., Bruce, J., Bowling, M., Veloso, M.: Stp: Skills, tactics and plays for multi-robot control in adversarial environments. IEEE Journal of Control and Systems Engineering 219, 33–52 (2005) 5. Bruce, J., Zickler, S., Licitra, M., Veloso, M.: CMDragons 2007 Team Description. Technical report, Tech Report CMU-CS-07-173, Carnegie Mellon University, School of Computer Science (2007) 6. G¨ unther, C.W., van der Aalst, W.M.P.: A generic import framework for process event logs. In: Eder, J., Dustdar, S. (eds.) BPI 2006. LNCS, vol. 4103, pp. 81– 92. Springer, Heidelberg (2006) 7. Han, K., Veloso, M.: Automated robot behavior recognition (2008) 8. Rozinat, A., van der Aalst, W.M.P.: Decision mining in proM. In: Dustdar, S., Fiadeiro, J.L., Sheth, A.P. (eds.) BPM 2006. LNCS, vol. 4102, pp. 420–425. Springer, Heidelberg (2006) 9. Vail, D., Veloso, M., Lafferty, J.: Conditional random fields for activity recognition. In: AAMAS 2007: Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems, pp. 1–8. ACM, New York (2007) 10. Weijters, A.J.M.M., van der Aalst, W.M.P.: Rediscovering Workflow Models from Event-Based Data using Little Thumb. Integrated Computer-Aided Engineering 10(2), 151–162 (2003) 11. Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005)

Altruistic Relationships for Optimizing Task Fulfillment in Robot Communities Christopher M. Clark, Ryan Morton, and George A. Bekey

Abstract. This paper introduces the concept of a multi-robot community in which multiple robots must fulfill their individual tasks while operating in a shared environment. Unlike typical multi-robot systems in which global cost functions are minimized while accomplishing a set of global tasks, the robots in this work have individual tasks to accomplish and individual cost functions to optimize (e.g. path length or number of objects to gather). A strategy is presented in which a robot may choose to aid in the completion of another robot’s task. This type of ”altruistic” action leads to evolving altruistic relationships between robots, and can ultimately result in a decrease in the individual cost functions of each robot. However, altruism with respect to another robot must be controlled such that it allows a relationship where both robots are altruistic, but protects an altruistic robot against a selfish robot that does not help others. A quantitative description of this altruism is presented, along with a law for controlling an individuals altruism. With a linear model of the altruism dynamics, altruistic relationships are proven to grow when robots are altruistic, but protect an altruistic robot from a selfish robot. Results of task planning simulations are presented that highlight the decrease in individual robot cost functions, as well as evolutionary trends of altruism between robots.

1 Introduction As the number of mobile robots increases within our homes, industry, and for scientific exploration, there is an increase in the number of situations where many robots will have to work together within a common workspace. Such situations will promote the necessity for a peaceful and constructive Multi-Robot Community (MRC). Christopher M. Clark, Ryan Morton, and George A. Bekey Computer Science, California Polytechnic State Univ., San Luis Obispo, CA, 93407 USA e-mail: [email protected],[email protected],[email protected]

262

C.M. Clark et al.

In this work, we depart from the situation in which a group of robots are designed, owned, or operated by a single individual or organization with the purpose of achieving a common goal. Here, an MRC consists of robots that may be designed, owned, or operated by several individuals or organizations that may have different goals. Robots may be designed to act in their own best interest and accomplish their own goals without concern of the goals of other robots in the community. That is, robots in an MRC may be selfish. This work proposes that robots in an MRC can achieve their goals more efficiently through the use of altruism. We define an altruistic robot as one which assists others in the attainment of their goals even if such actions may be harmful to itself. Specifically, if robots are willing to perform the tasks of other robots while decreasing their own efficiency in the short run, large gains in individual and global efficiency can be made over long time horizons. As an example, consider two autonomous robots commissioned to pick up courier mail from around the city. While these robots might be owned and operated by different and possibly competing organizations, it may be in the best interest of both robots for them to accomplish each other’s tasks when appropriate. That is, if robot A’s pickup location is closer to robot B, then robot B should consider performing this pickup for robot A. This would be an altruistic action for robot B because it reduces efficiency without accomplishing any of its own pickups. However, robot A might later reciprocate this altruistic action, thereby building an altruistic relationship as a result of which both A and B may complete their tasks more rapidly or with less total distance traveled. In the next section, related work is briefly covered. Section 3 defines a framework for an MRC as well as the problem being addressed in this paper. Section 4 presents a method for robots to control their altruism within the MRC framework, highlighting the ability for robots to create altruistic relationships while protecting them against purely selfish robots. In section 5, simulation results are presented. Finally, conclusions and future work are provided in section 6.

2 Background Multi-Robot Systems (MRS) have been an active area of robotics research [2], due to the several potential advantages over single robot systems. Namely, they offer the possibility for greater spacio-temporal sampling, force multiplication, and robustness to failure. Advancements in the areas of MRS mission planning, MRS motion planning, MRS localization, MRS mapping, and the most related subject of task allocation have occurred over the last two decades.

2.1 Task Allocation Of particular relevance to this work is the MRS task allocation problem, in which the MRS must accomplish a set of tasks characterized by their geographic

Altruistic Relationships for Optimizing Task Fulfillment

263

location. The problem is to determine the optimal assignment of task points to robots, along with the optimal sequence for robots to visit these task points that minimizes the time to visit all task points. This is a variation of the MultiTraveling Salesperson Problem (MTSP), a problem with many applications but no polynomial time solution (e.g. [12]). Regardless, many good heuristic driven methods have been developed that yield sub-optimal solutions. One popular method of assigning tasks to robots in an MRS is to use a Market-Based Auction approach [6]. In this method, tasks are auctioned off to robots with the highest bids. Bids are typically based on the ability of the robot to accomplish the task, while considering the additional cost of traveling to the task site. While this method is not guaranteed to find optimal solutions, it is efficient and can lead to near optimal solutions.

2.2 Altruism In the literature on robotics there are extensive treatments of cooperation among robots, but little mention of altruistic behaviors. Cooperation may in fact be altruistic, but it is generally not described in those terms in the literature. Examples of work involving cooperation are [4], [3], [10], and [8]. Work directly involving altruism includes that of [11] and [7] who describe robot behaviors in terms of a satisfaction index and transmission/reception of signals from other robots. A robots progress in a given task can be measured by its satisfaction in the task, which corresponds to the fitness or performance index indicated above. Thus, a robot needing help with a task may emit an attraction “please help me” signal. Lucidarme et. al. [7] propose an altruism vector based on the satisfaction index of a robot and the signals emitted by other robots; a given robot decides on altruistic behavior based on the magnitude of this vector. Similarly, [1] describes a software architecture for robot colonies based on robot tropisms, defined as their likes and dislikes. Reinforcement of particular behaviors strengthens them in future scenarios. Here too a robot can call for help to other robots when it needs assistance in moving heavy objects beyond its capability. The emergence of cooperative behaviors has been studied extensively in game theory, under the name Prisoners Dilemma, e.g.[9]. However, while the winning strategies in this situation call for cooperation, altruism is not in the discussions known to us.

2.3 Reputation Management Reputation Management (RM) occurs when an agent evaluates the actions of other agents, forms opinions about those agents, and then uses these opinions to adjust its own actions. The field of RM involves analysis of such processes with applications ranging from interpersonal relationships to economics. A survey or RM with an emphasis on its application to the online marketplace is

264

C.M. Clark et al.

presented in [5]. A related example can be found in [13], where RM is applied to the general area of ”Electronic Communities”. This work demonstrates the positive development of altruistic relationships in which the trust of other agents can be built up over time. This has close similarities to the application within an MRC, but uses the trust to assess the quality of information from other agents. Here, we use this trust to determine if robots should be altruistic to one another, thereby improving individual performance.

3 Multi-robot Communities A Multi-Robot Community C = {r1 , r2 , ..., rN } is a set of N robots that can interact through some shared workspace W . In this community, each robot ri will have a set of Li individual tasks to accomplish described by the possibly dynamic set Ti = {ti1 , ti2 , ..., tiLi }. Such tasks may include taking measurements, picking up materials, placing objects, etc. A task tij is considered to be completed if any of the robots within C visits the task location. Therefore, once tasks are assigned, each robot ri plans a sequence Si of task locations to visit that minimizes path length. Si = {tkn , tlo , ..., tmp }

(1)

where indices k, l, m, n, o, p are arbitrary at this point to reflect the possibility that robot ri ’s task sequence Si may include any of the nth , oth , or pth tasks belonging to any of the k th , lth , or mth robots within the community. While different task sequencing algorithms may be used, it should be clear that the effectiveness of the task allocation is related to how close the planned sequences are to the optimal task sequence Si∗ . The cost incurred to accomplish a task sequence is calculated based on the 2 dimensional euclidean distance dkn,lo between two task locations tkn and tlo . Hence, a candidate for each robot’s personal cost function is  dkn,lo (2) Ji = Ji (Si ) = wi tkn ,tlo ∈Si

Where wi has units s·m−1 and for this paper has value 1. Note that, instead of eq. (2), several other cost functions could be used within the altruistic MRC framework that follows. The global cost function typically used to characterize performance for task allocation in an MRS would be Jglobal =

N 1  Ji N i

(3)

This cost function can be used to measure performance of an MRC. However, individual robots participating within an MRC will most likely use only their individual cost function described in eq. (2). Hence, controllers should be designed with such cost functions in consideration.

Altruistic Relationships for Optimizing Task Fulfillment

265

3.1 Task Fulfillment in MRC A common approach taken for optimizing task allocation is to implement a market-based auction system in which robots place bids for tasks (e.g. [6]). Through inter-robot communication, each robot ri can communicate its bid on each task as it is auctioned. The task will be awarded to the robot with the lowest bid for that task. A robot’s bid on a particular task will be calculated based on the ability to complete the task in minimal time. For example, if τij is the time for robot ri to accomplish task j and all other tasks awarded in previous bidding wars, then the bid bij can be set to equal τij . In this work, each robot ri can auction any of its own tasks from Ti . Other robots can choose to bid on robot ri ’s tasks, thus acting in an altruistic manner if they win the bid and complete the task. This choice is based on the level of altruism one robot may have towards another robot. To be precise, we define the variable αij ∈ [0, ∞] as the level of altruism robot ri has towards robot robot rj . A robot ri will bid on other robot rj ’s task tjk according to: bi,jk =



Ji (Si ∪ tjk ) ∞

if αij > Ji (Si ∪ tjk ) − Ji (Si ) else

(4)

The goal is to control this value of α such that robots will behave altruistically towards one another. This will lead to robots doing each other’s tasks when more efficient, thereby decreasing cost functions Ji . However, since robots are trying to minimize their own cost, they have incentive to act in a selfish manner by maximizing the number of their tasks completed by other robots and minimizing the number of tasks they themselves complete. Hence it is not always beneficial to simply set αij to some constant that ensures altruistic behavior (e.g. setting αij = ∞ ). Instead, robots can adapt αij on-the-fly in response to the actions of other robots, or more specifically in response to the complimentary altruism, αji . When such dynamic altruistic behavior is allowed, analysis of the altruistic dynamics within an MRC is required. To do so, a standard discretized linear time-invariant state space model is proposed: αt+1= Aαt + But

(5) T

In this equation, α = [α12 α21 α13 α31 ... αij ] is the state vector, t is the time step, A = I N xN is the state matrix, B = 1 is the input matrix, and u = [u12 u21 u13 u31 ... uij ]T is the control input. While this implies a simple model, different values for A and B are possible and may require more complicated control laws.

266

C.M. Clark et al.

4 Altruism Controller Design This section provides a controller for setting u in equation (5), that allows for mutually altruistic relationships to form. Before proceeding to the design of control laws for αij , it should be noted that the system assumes: 1) all robots can reliably communicate with one another to conduct auctions and bidding, and 2) all robots can either directly measure the altruistic nature αji of other robots. This second assumption implies that individual cost functions have the same units and can be compared. In the control strategy proposed, each robot ri will try to set its altruistic nature αij towards another robot to be that which the other robot rj has toward it. This is accomplished through the following proportional control law: uij = Kij (αji + ǫij − αij )

(6)

The control gain Ki > 0 determines the rate at which αij approaches the desired value of αji + ǫij . The first term in this desired value is the altruism that robot rj has towards robot ri . The second term, ǫij > 0, indicates how much more altruistically robot ri will act towards robot rj . It is important to note that ǫij is used to allow altruism to grow between two robots in that if both robots use such a control law, their complimentary values αij and αji will grow throughout time. Consider the resulting state model of the altruistic relationship between 2 robots that can occur, regardless of the other robots in C. Note that without losing generality, it is assumed K12 = K21 = K and ǫ12 = ǫ21 = ǫ. ⎞⎛ ⎛ ⎛ ⎞ ⎞ α12 1−K K K α12 ⎝ α21 ⎠ = ⎝ K 1 − K K ⎠ ⎝ α21 ⎠ (7) 0 0 1 ǫ ǫ t+1 t The stability of the system can be evaluated through a coordinate transformation e1 = α12 − α21 . Given this transformation, the system can be restated as error dynamics: e1t+1 = (1 − 2K)e1t

(8)

Hence if |1 − 2K| < 1 the error dynamics will be stable and it follows that the error (α12 − α21 ) → 0 as t → ∞. Now, consider a desired rate of change of altruism to be Kǫ, then the error in the rate of change of altruism is: e2t+2 = (α12t+1 − α12t ) − Kǫ

(9)

Substituting the top row of eq. (7) into eq. (9) yields: e2t+2 = −Ke1t

(10)

Altruistic Relationships for Optimizing Task Fulfillment

(a)

(b)

267

(c)

Fig. 1 Task fulfillment in an MRC when for all i,j, the altruistic nature in (a) of robots αij = 0% and individual robot paths must span large portions of the workspace to visit individual task points. In (b), αij = 5% and when αij = 100% (c). With altruism, robots tend to create more localized paths that demonstrate increased effi ciency. Altruism are percentages of greatest distance a task can add to a robots path in the given workspace

Again if |1 − 2K| < 1, e1 → 0 as t → +∞. It then follows that e2 → 0 as t → +∞. More explicitly, the rate of change of altruism (α12t+1 − α12t ) stabilizes to Kǫ. Thus, for gain conditions 0 < K < 1, the mutual altruisms α12 and α21 will both match each other and grow over time. If, on the other hand, robot ri attempts to be altruistic towards a selfish robot rj , (i.e. αji = 0), then the state transition from eq. (7) reduces to:    α12 α12 1−K K (11) = 0 1 ǫ t ǫ t+1 In this case we can define the error to minimize with the transformation e3 = α12 − ǫ. Substituting top row of eq. (11) into this transformation yields error dynamics: e3t+1 = (1 − K)e3t

(12)

Hence if 0 < K < 1, then e3 → 0 as t → +∞, and it follows that the corresponding state α12 is stable. In fact, this behaves like a Proportional control system where ǫ is the desired state. To note, the αij controller presented above is dependent on the reciprocal altruism αji . If robot rj can not be trusted, then its altruism must be estimated based directly on its actions and any task point tik on which it bids. In this work, a conservative estimate is used:

α ˆ ij,t+1 =



max(ˆ αij,t , Jj (Sj ∪ tik ) − Jj (Sj )) min(ˆ αij,t , Jj (Sj ∪ tik ) − Jj (Sj ))

if robot j bids else

(13)

268

C.M. Clark et al.

5 Results To demonstrate the general effect on an MRC when robots create altruistic relationships, a simulated task fulfillment experiment was conducted in which 500 tasks were randomly created within a 6.4x6.4 m 2D workspace as shown in Fig. (1). Each task was randomly assigned to be within the task set of one of 8 robots operating in the space. After each of the 500 tasks are assigned, robots auction them off to all robots using an assumed wireless communication system. Robots will bid if they have sufficiently large altruism towards the auctioneer. The order of auctioning is random. For each run of the experiment, robots had fixed but equal values of altruism. Between experiments, the altruism was incremented by 1% up to 20%, then incremented by 10%. In Fig. (2), the average robot path cost in meters is plotted for these values of altruism. Note that the values for altruism are normalized with respect to the percentage of the maximum value defined as the greatest distance between two points in the workspace (i.e. the length of the diagonal connecting opposite corners). The path costs show a dramatic decrease when compared with the case with no altruism. The cost plateaus where additional tasks typically won’t cost a robot more than 20% of the largest distance a task point can occur on path cost for the given workspace. This can be observed in Fig. (1) as well, where in (a) no altruism results in paths that cross one another and the entire environment. However, in (c) where 100% altruism is used, paths are more localized and the personal cost functions decrease to the effect that the global cost function is also minimized. To demonstrate the effects of the control law, consider the case where two robots start with random values for altruism with respect to each other (e.g. 0%, 18%). As shown in Fig. 3(a), the altruism between the two robots first converges and then grows with time (i.e. as new tasks are auctioned). It can be noted that the rising slope calculated from Fig. 3(a) is 1/16 which matches the expected value of Kǫ = (0.25)(0.25).

Fig. 2 Potential effects of increased altruism within an MRC

Altruistic Relationships for Optimizing Task Fulfillment

(a)

(b)

(c)

(d)

269

Fig. 3 Stability of altruism within an MRC. In (a), both robots use the altruistic controller but with different initial values (0% and 18%). In (b), only one robot uses the controller while the other robot maintains a fixed value of 0. In (c) and (d), the altruism must be estimated by robots

When robots are acting selfishly, the controller invoked by the altruistic robot is stable, (see Fig.3(b)). The altruism towards such selfish robots reaches equilibrium at the point ǫ greater than 0. Hence, the altruistic robot is still protected against selfish robots. Note that increasing the gain K increases the speed of convergence. In (c) and (d), cases where the altruism of other robots are estimated using equation (13) are presented. In (c), robot 1 has altruism fixed at 100%. Robot 2’s altruism towards 1 starts at 100% and converges to the maximum bid used by robot 1. In (d), robot 1 has altruism fixed at 0, it can be seen that robot 2’s altruism converges towards ǫ.

6 Conclusions and Future Work This paper presented the idea of a Multi-Robot Community in which several robots sharing a common workspace are attempting to complete their individual tasks. It was shown that altruistic actions, where robots assist with each other’s tasks, can lead to decreased path costs for individual robots.

270

C.M. Clark et al.

While a controller was presented that can set altruism such that altruistic relationships can evolve between two altruistic robots and still protect against selfish robots, the real contribution comes from the idea of analyzing the stability of altruism as a linear time-invariant system. Future controllers can be designed and analyzed in a similar fashion. Considerable work is still required, especially to resolve the assumptions listed above. In the future it is hoped that better estimation of another robot’s altruism can be achieved, selfish robots that have variable values for alpha are addressed (e.g. gaming robots), situations with uneven task distribution are considered (5 tasks for one robot 100 for another), and practical implementation and experimentation may occur.

References 1. Agah, A., Bekey, G.A.: Tropism-based cognition: a novel software architecture for agents in colonies. Journal of Experimental and Theoretical Artificial Intelligence 9, 393–404 (1997) 2. Arkin, R.C., Bekey, G.A.: Robot Colonies. Springer, Heidelberg (1997) 3. Arkin, R.C., Balch, T.: Cooperative multiagent robotic systems. In: Kortenkamp, D., Bonasso, R.P., Murphy, R. (eds.) Artificial Intelligence and Mobile Robots, pp. 277–295. AAAI Press/MIT Press (1998) 4. Brooks: A robust layered control system for a mobile robot. IEEE Journal of Robotics and Automation RA-2, 14–23 5. Dellarocas, C.: Reputation Mechanisms. In: Hendershott, T. (ed.) Handbook on Information Systems and Economics, pp. 154–165. Elsevier, Amsterdam (2006) 6. Goldberg, D., Cicirello, V., Dias, M.B., Simmons, R., Smith, S., Stentz, A.: Task Allocation Using a Distributed Market-Based Planning Mechanism. In: International Conference on Intelligent Agents (2003) 7. Lucidarme, P., Simonin, O., Liegeois, A.: Implementation and evaluation of a satisfaction/altruism based architecture for multi-robot systems. In: Proc. IEEE Int. Conf. on Robotics and Automation, pp. 1007–1012 (2002) 8. Mataric, M.: Behavior based control: Examples from navigation, learning and group behavior. Jour. of Experimental and Theoretical Artificial Intelligence 9(2-3), 67–83 (1997) 9. Nowak, M.A., May, R.M., Sigmund, K.: The arithmetics of mutual help. Scientific American 272(6), 76–81 (1995) 10. Parker, L.E.: ALLIANCE: An architecture for fault tolerant multirobot cooperation. IEEE Trans. on Robotics and Automation 14(2), 220–240 (1998) 11. Simonin, O., Ferber, J.: Modeling self-satisfaction and altruism to handle action and reactive cooperation. In: SAB 2000 Proceedings, pp. 314–323 (2000) 12. Sofge, D., Schultz, A., De Jong, K.: Evolutionary Computational Approaches to Solving the Multiple Traveling Salesman Problem Using a Neighborhood Attractor Schema. In: Applications of Evolutionary Computing - EvoWorkshops (2002) 13. Yu, B., Singh, M.P.: A Social Mechanism of Reputation Management in Electronic Communities. In: Cooperative Information Agents (CIA) 2000, pp. 154– 165. Springer, Heidelberg (2000)

Part V

Multi-robot Cooperation

Robotic Self-replication, Self-diagnosis, and Self-repair: Probabilistic Considerations Gregory S. Chirikjian

Abstract. This paper reviews issues related to two topics: (1) “self-replicating robots,” i.e., robots that can make copies of themselves from spare parts; and (2) robotic teams that can “self-diagnose,” which is an essential attribute that will lead to the ability of robots to autonomously and cooperatively repair other robots. In order to quantify the robustness of such robots, statistical measures of the degree of environmental uncertainty that they can handle must be computed. The entropy of the set of all possible arrangements (or configurations) of spare parts in the environment is just such a measure. In the team diagnosis and repair scenario, a team of robots works cooperatively to perform tasks. During this process, all members of the team diagnose each other to determine if there is any abnormal behavior in any team member. When there is, the members of the team can, in principle, cooperate to either repair the malfunctioning team member or disassemble it for use as spare parts. The algorithmic side of this problem is also statistical in nature, and is briefly reviewed here.

1 Introduction The concept of a robot that can build a copy of itself from spare parts is more than half a century old. The first theoretical work on self-reproducing machines was performed by John von Neumann in the 1950s [22]. The earliest physical demonstrations of replicating machines were presented in [14] and [24]. In [24] simple units self-reproduced via assembly of passive mechanical elements under external vibration. The system in [14] worked in a similar way, but used electric carts operating on an oval track instead of mechanical components confined in a box. G.S. Chirikjian Department of Mechanical Engineering Johns Hopkins University Baltimore, MD 21218, USA e-mail: [email protected]

274

G.S. Chirikjian

Experimental work related to robotic replication falls loosely into four categories: directed replication via module assembly (see [18, 19] and references therein), directed replication via fabrication [31], and self-assembly of randomly agitated modules (e.g., [24, 21]). The area of self-reconfigurable modular robots [3, 27, 28, 29, 30] is related to this in that complex robotic systems are composed of less complex modules. The area of cellular robotic systems began with the work of Fukuda et al [12]. This exciting area of research has continued to the current day, and includes recent hardware implementations such as the MTRAN series of robots [15, 16] and the ATRON [23, 9]. The ideas reviewed here to quantify the difficulty of self-replication use Sanderson’s parts entropy concept [25], and draw heavily on group-theoretic methods. In particular, the connection between symmetry of parts and assemblies, and the connection with how much change in entropy occurs during assembly/disassembly is analyzed with the end goal of designing better components and assemblies. The use of symmetry groups in assembly operations and planning has been studied extensively in the past [20], as has the development of algorithms to reduce disorder in the orientation of parts without using sensors [10]. One of the primary applications of self-replicating robots is as potential means for space development and exploration [11, 8]. In a more down-to-earth setting, issues in robotic systems that can repair themselves or other robots has the potential to make robotic systems more robust. Work on self-repair, which is closely related to the modular self-reconfigurable systems in [3, 28, 29, 30] (and the seminal work by Kokaji and Fukuda from the 1980s that preceded this), is closely related to selfreplication. In recent work, the author’s group has been exploring statistical methods to be employed in algorithms for teams of mobile robots that are not physically connected, that can diagnose and repair members of the team [17]. This is somewhat different than the strategies employed for the fault detection and diagnosis in industrial systems, as described in [4, 13].

2 Parts Entropy and the Difficulty of Assembly Tasks In the field of assembly automation it has long been known that the design of machines that assemble parts should take advantage of part geometries [2]. And robotic systems with minimal sensing can achieve goals if information about the parts and environment are known [10]. Systems that shake or otherwise randomize part positions and orientations (i.e., “poses”) allow a collection of parts to sample the ensemble of all possible poses. Almost 25 years ago, Sanderson quantified the concept of part disorder by defining the concept of “parts entropy” [25]. In his analysis, he considered the tractable problem of noninteracting parts, thereby focusing on individual part entropies. In contrast, in [5] the concept of parts entropy is extended to the case of multiple interacting parts. Various issues associated with computing the entropy of ensembles of configurations of parts with excluded-volume constraints are reviewed here. The rapid computation of excluded-volume effects using the “Principal

Robotic Self-replication, Self-diagnosis, and Self-repair

275

Kinematic Formula” from the field of Integral Geometry is illustrated. References in the English language on this topic include [26] Let G = SE(n) for n = 2 or 3 denote the group of rigid-body motions in the plane or in space, and let g denote an arbitrary rigid-body motion in G. Let f (g) be a  probability density function on G. That is, G f (g)dg = 1 where dg is the natural (bi-invariant) integration measure for G, as explained in [7]. The entropy of one isolated part was defined by Sanderson as [25, 7] S=−



f (g) log f (g)dg.

G

This is a measure of “how disordered” the set of all possible positions and orientations of a rigid part can be. It need not be the case that f (g) is uniform, and it need not be the case that it is even smooth. For example, we may want to study the entropy of a polyhedral object (which is spatial, and the motions of which are described as elements of SE(3)) in terms of all possible ways that it can sit stably on a table. If there are two parts in an environment, then the combinations of all possible ways that they can be simultaneously positioned and oriented without ′ overlap   ′ is denoted here as f (g1 , g2 ) It is a probability normalized such that G G f (g1 , g2 )dg1 dg2 = 1 and the corresponding parts entropy is S12 = −

 

G G

2

f ′ (g1 , g2 ) log f ′ (g1 , g2 )dg1 dg2 ≤ − ∑



i=1 G

fi′ (gi ) log fi′ (gi )dgi . (1)

where fi′ (g1 ) = G f ′ (g1 , g2 )dgi . Equality in (1) holds if and only if f ′ (g1 , g2 ) = f1 (g1 ) f2 (g2 ). Therefore, treating the distributions of two parts separately without regard to their interaction, will always overestimate the total parts entropy. While the discussion below will be limited to one and two parts, the extension to multiple parts follows in a natural way. 

2.1 The Principal Kinematic Formula Given n parts, the ith of which is free to be placed arbitrarily in an environment with frequency of occurrence given by fi (gi ), the entropy will be bounded by S′ ≤ ∑ni=1 Si where Si is the entropy of the ith part computed independently (i.e., as if that were the only part). If, however, the environment is very cluttered and there is not a significant amount of free space, this bound will not be tight, and the entropy of the joint distribution of parts will have to be computed: S′ = −



Gn

f ′ log f ′ dg1 · · · dgn where

and f ′ = f ′ (g1 , g2 , ..., gn ).



Gn

=



G

···



G

(2)

276

G.S. Chirikjian

n Whereas in the independent case we have  that f (g1 , g2 , ..., gn ) = ∏i=1 fi (gi ) where for each i = 1, ..., n the normalization G fi (gi )dgi = 1 holds. In general this simple form is not realistic and needs to be augmented to reflect the excluded volume of parts. Suppose that we have two convex bodies, B1 and B2 , viewed as subsets of IRn . Recall that a body is convex if the line segment connecting two points in the body is contained in the body for all possible choices of pairs of points. Another way to say this is that if for any x, y ∈ B, then tx + (1 − t)y ∈ B for all t ∈ [0, 1]. The indicator function on any measurable body, B, (not necessarily convex and perhaps not even connected) is defined by:  1 if B = 0/ i(B) = 0 for B = 0/

Note that if g is some group of operations that acts on B without shrinking it to the empty set, then i(g · B) = i(B). The indicator function is one of many functions on a body that is invariant under rigid-body motion. Others include the volume of the body, the surface area (or perimeter in the two-dimensional case). Let B1 be stationary, and let B2 be mobile. Let us denote g ∈ G (where G = SE(n) is the group of rigid-body motions for bodies contained in IRn ). Then, by definition, gB = {g · x|x ∈ B}, where if g = (A, a) is the rigid-body motion with rotational part A ∈ SO(n) and translational part a ∈ IRn , then the action of G = SE(n) on IRn is g · x = Ax + a. “Counting up” all values of g for which an intersection occurs is then equivalent to computing the integral  J=

G

i(B1 ∩ gB2 )dg.

(3)

A somewhat amazing result is that the integral J can be computed exactly using only elementary geometric properties of the bodies B1 and B2 without actually having to perform an integration over G. Theorem (Blaschke, [1]): Given planar convex bodies B1 and B2 , then J evaluates as:  i(B1 ∩ gB2 )dg = 2π [A(B1 ) + A(B2)] + L(B1 )L(B2 ). (4) SE(2)

where A(·) is the area and L(·) is the perimeter of the body. (See [1, 26] for proof.) The entropy of a single isolated part under these conditions is then S f = logV. In contrast, the total volume in SE(n) that is available for part #2 to move if part # 1 is fixed in the environment, thereby limiting the range of possible motions of part #2, and so the entropy of part #2 when part #1 is fixed will be S f ′ = logV ′ where V ′ ≈ V −



SE(n)

i(B1 ∩ gB2 )dg

Robotic Self-replication, Self-diagnosis, and Self-repair

277

This assumes that the parts are small relative to the environment and that boundary effects of the environment can be neglected.

2.2 Symmetry Considerations In the previous section the fact that SE(n) is a Lie group is only important because it allows for an appropriate way to shift and integrate over functions. However, when symmetry considerations are taken into account, other aspects of group theory are required. Here the relationship between part symmetry, group theory and the relationship to part entropy is explored. A review of the basic definitions of group theory (groups, subgroups, group actions, etc., are known by the reader) can be found in [7]. Given a subgroup H ≤ G, and any element g ∈ G, the left coset gH is defined as gH = {g ◦ h|h ∈ H}. (This should not be confused with gB, which denotes a rigidly moved copy of the body B, which is not a subgroup of G). Similarly, the right coset Hg is defined as Hg = {h ◦ g|h ∈ H}. For example, any rigid part that can move in three-dimensional space is acted on by the group of rigid-body motions G = SE(3). If the part has symmetry, then G is divided into cosets defined by ‘equivalent’ poses of the object relative to its symmetry group H. H can be a continuous subgroup (for instance if the part is described as a surface of revolution), or it can be finite (for instance if it has a four-fold rotational symmetry). The set of all left(or right) cosets is called the left(or right) coset space, and is denoted as G/H (or H\G). Here we consider how symmetries in parts affect the computation of entropy. In particular, these symmetries are reflected in the function F(g) = − f (g) log f (g), and decompose the domain of the integral into identical subregions.        F(g)d(g) = F(g ◦ h)d(h) d(gH) = F(h ◦ g)d(h) d(Hg) G

G/H

H

H\G

H

where g is a representative of the coset gH and of the coset Hg. The reason why such formulas are important is because the symmetry of parts, and the the symmetry of the assemblages into which parts are placed during a construction/repair process, effectively mean that these parts are easier to insert in the correct way. And this is reflected in the entropy calculation, as described in [6].

3 Team Diagnosis: Maneuvers, Observations and Fault States Imagine a team consisting of N autonomous robots. For the sake of simplicity assume that these robots are all identical, and that each robot can be assembled from m subsystems (or modules) by a subset of N ′ < N of the other robots. Also assume that each robot can be decomposed into its constituent subsystems by N ′ of the other robots. The mechanical difficulty involved in the robotic construction and repair of other robots can be quantified using the tools described in the previous section. However, another issue to be addressed is that of determining when N ′ robots

278

G.S. Chirikjian

should act to repair another robot in the team. This is the team diagnosis problem which is addressed in [17], and reviewed here. Imagine that each robot in the team can be characterized by a finite number of system states. Contained within this set of system states is a finite number of fault states that indicate the need for other robots to either rescue, repair or decommission an individual robot. Denote the set of system states as {ξ1 , ξ2 , ..., ξn }. Some of these states might correspond to a functional robot that is prevented from functioning due to its interactions with the environment, and others might be due to internal malfunctioning of the robot. For example, if a robot has one wheel stuck in the mud, then that robot may not be in need of repair, but it may need to be rescued. Or, if a motor that turns one wheel burns out then that wheel assembly can either be replaced if another member of the team is carrying the correct spare part, or the whole robot can be cannibalized for spare parts. Of course, the system state corresponding to the robot properly performing its duties is also possible, in which case the result of the diagnosis should be to not perform a repair operation. Unfortunately, system states are difficult to assess directly. However, a combination of internal diagnostics and external observations can be pooled to arrive at estimates of the state of the robot. After a diagnostic command is issued to an individual robot, external observations by other robots can be used to assess performance. Let X be the system inputs for the robot being diagnosed. For example, the robot might apply a set of voltages or currents over a period of time from t0 to tF . This period of time can be divided into p subsets, and so the system input can be viewed as the vector X = [X1 , ..., X p ]T where Xi is an observation over the time interval from t0 +(i−1)(tF −t0 )/p to t0 +i(tF −t0 )/p. Discrete observations obtained by the other robots can be made on each of these p periods. This is denoted as the column vector Y = [Y 1 , ...,Y p ]T . For example, the relative position and orientation of the robot measured during each of these time periods as it undergoes a diagnostic maneuver can be assessed. Then Y i (t) ∈ G for t0 + (i − 1)(tF − t0 )/p ≤ t ≤ t0 + i(tF − t0 )/p is a trajectory in the group of rigid-body motions (which for a robot moving in the plane is SE(2)). (The reason for using superscripts on Y rather than subscripts will become clear momentarily.) Unfortunately, there is rarely a one-to-one correspondence between the value of the observation Y and any particular system state ξi . For example, external observations might not distinguish between a robot with a burnt out motor in a wheel assembly or a wheel that is stuck in the mud. However, for each system input Xi , the set of systemstates {ξ1 , ξ 2 , ..., ξn } can be divided into a smaller set of classes of system states ξˆ1i , ξˆ2i , ..., ξˆqi by the observation Y i where 1 ≤ q < n. That is, if a diagnostic command is issued that results in the robot executing a diagnostic maneuver (i.e., the motion that the robot undergoes), then each observation Y i , can be used to assess which class of possible states that the observed robot is in. In particular, Y ji denotes an observation consistent with the class of system states ξˆ i . The diagnostic j

command Xi corresponds to a trajectory of the robot that can be r points equally spaced in time. The distribution of relative poses of the robot at the end of each of

Robotic Self-replication, Self-diagnosis, and Self-repair

279

these sub-trajectories relative to the beginning can then be compared to observations of each class of system states obtained during a training/identification procedure. If the diagnostic maneuver is constructed carefully enough, then the following properties will allow for the winnowing down of the observations to arrive at the actual system state [17]: ⎧ ξj ⎪ ⎪ ⎪ ⎪ q q ⎨

ξˆ ji = {ξ1 , ξ2 , ..., ξn } , ξˆ ji ξˆki = 0, / ξˆ j11 ∩ ξˆ j22 ∩ · · · ∩ ξˆ jpp = or ⎪ ⎪ j=1 j,k=1 ⎪ ⎪ ⎩ 0/

where ξ j = ξ j ( j1 , j2 , ..., j p ; 1, 2, ..., p). That is, because of these properties, if classes of system states are identified from individual trajectories within the diagnostic maneuver, then the “completeness” of the maneuver will allow for extraction of the actual system state. A detailed probabilistic procedure for doing this is outlined in [17]. In that procedure, the observations Y ji were poses in the plane, i.e, triplets of the form (xij , yij , θ ji ). This information can be viewed as g(Y ji ) ∈ SE(2), and so the diagnostic procedure can be viewed as obtaining the best match of observed probability densities in position and orientation f (g(Y ji )), with probability densities obtained during an identification/testing phase of the form f (g; ξˆ i ). The best match between these serves as a method of j

establishing (probabilistically) which class of system states the robot is in.

4 Conclusions The difficulty in designing robots that self-replicate and self-repair is due to a number of issues including: (1) the amount of “parts entropy” that the system is able to reduce via an assembly procedure given its sensing capabilities; and (2) the ability to properly diagnose the state of a robot that is being replicated or repaired. In this paper, mathematical formulations that quantify these problems are presented. These formulations complement the clever hardware and algorithms that are being design by a number of research groups. Acknowledgements. The author wishes to acknowledge Matthew Moses, Michael Kutzer, and Kiju Lee for discussions related to this work.

References 1. Blaschke, W.: Vorlesungen u¨ ber Integralgeometrie. Deutscher Verlag der Wissenschaften, Berlin (1955) 2. Boothroyd, G.: Assembly Automation and Product Design, 2nd edn. CRC Press, Boca Raton (2005)

280

G.S. Chirikjian

3. Bulter, Z., Murata, S., Rus, D.: Distributed replication algorithm for self-reconfiguring modular robots. In: Proceedings of 6th International Symposium on Distributed Autanomous Robotic Systems (DARS 2002), Fukuda, Japan, pp. 25–27 (June 2002) 4. Chiang, L.H., Russell, E.L., Braatz, R.D.: Fault Detection and Diagnosis in Industrial Systems. Springer, London (2001) 5. Chirikjian, G.S.: Parts Entropy and The Principal Kinematic Formula. In: IEEE Conference on Automation Science and Engineering (CASE 2008), Washington D.C (August 2008) 6. Chirikjian, G.S.: Parts Entropy, Symmetry, and The Difficulty of Self-Replication. In: ASME Dynamic Systems and Control Conference (DSC 2008), Ann Arbor, Michigan, October 20-22 (2008) 7. Chirikjian, G.S.: Stochastic Models, Information Theory, and Lie Groups, vol. 2. Birkh¨auser, Basel (to appear, 2009) 8. Chirikjian, G.S., Zhou, Y., Suthakorn, J.: Self-Replicating Robots for Lunar Development. IEEE/ASME Transactions on Mechatronics 7(4), 462–472 (2002) 9. Christensen, D.J., Brandt, D., Støy, K.: Towards artificial ATRON animals: scalable anatomy for self-reconfigurable robots. In: Robotics Science and Systems: Workshop on Modular Self-Reconfigurable Systems, Philadelphia (August 2006) 10. Erdmann, M.A., Mason, M.T.: An Exploration of Sensorless Manipulation. Journal of Robotics and Automation 4(4), 369–379 (1988) 11. Freitas, R.A., Merkle, R.C.: Kinematic Self-Replicating Machines (Landes Bioscience) (2004) 12. Fukuda, T., Nakagawa, S., Kawauchi, Y., Buss, M.: Self organizing robots based on cell structures–CEBOT. In: Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems (IROS), pp. 145–150 (November 1988) 13. Gertler, J.J.: Fault Detection and Diagnosis in Engineering Systems. Marcel Dekker, New York (1998) 14. Jacobson, H.: On Models of Reproduction. American Scientist 46, 255–284 (1958) 15. Kamimura, A., Kurokawa, H., Yoshida, E., Murata, S., Tomita, K., Kokaji, S.: Automatic locomotion design and experiments for a Modular robotic system. IEEE/ASME Transactions on Mechatronics 10(3), 314–325 (2005) 16. Kurokawa, H., Tomita, K., Kamimura, A., Murata, S., Terada, Y., Kokaji, S.: Distributed Metamorphosis Control of a Modular Robotic System M-TRAN. In: Gini, M., Voyles, R. (eds.) Distributed Autonomous Robotic Systems 7, pp. 115–124. Springer, Heidelberg (2006) 17. Kutzer, M.D.M., Armand, M., Lin, E., Scheidt, D., Chirikjian, G.S.: Toward Cooperative Team-Diagnosis in Multi-robot Systems. International Journal of Robotics Research 27, 1069–1090 (2008) 18. Lee, K., Moses, M., Chirikjian, G.S.: Robotic Self-Replication in Partially Structured Environments: Physical Demonstrations and Complexity Measures. International Journal of Robotics Research 27(3-4), 387–401 (2008) 19. Lee, K., Chirikjian, G.S.: Robotic Self-Replication from Low- Complexity Parts. IEEE Robotics and Automation Magazine 14(4), 34–43 (2007) 20. Liu, Y., Popplestone, R.J.: Symmetry Groups in Analysis of Assembly Kinematics. In: ICRA 1991, Sacramento, CA, April 1991, pp. 572–577 (1991) 21. Napp, N., Burden, S., Klavins, E.: The statistical dynamics of programmed selfassembly. In: ICRA 2006, pp. 1469–1476 (2006) 22. von Neumann, J., Burks, A.W.: Theory of Self-Reproducing Automata. University of Illinois Press (1962)

Robotic Self-replication, Self-diagnosis, and Self-repair

281

23. Østergaard, E.H., Kassow, K., Beck, R., Lund, H.H.: Design of the ATRON lattice-based self-reconfigurable robot Autonomous Robots 21(2), 165–183 (September 2006) 24. Penrose, L.S.: Self-reproducing machines. Scientific American 200(6), 105–114 (1959) 25. Sanderson, A.C.: Parts Entropy Methods For Robotic Assembly System Design. In: Proceedings of the 1984 IEEE International Conference on Robotics and Automation (ICRA 1984), vol. 1, pp. 600–608 (March 1984) 26. Santal´o, L.: Integral Geometry and Geometric Probability. Cambridge University Press, Cambridge (2004) (originally published in 1976 by Addison-Wesley) 27. Shen, W.-M., Salemi, B., Will, P.: Hormone-inspired adaptive communication and distributed control for CONRO self-reconfigurable robots. IEEE Trans. Robotics Automat. 18(5), 700–712 (2002) 28. Tomita, K., Murata, S., Kurokawa, H., Yoshida, E., Kokaji, S.: Self-assembly and selfrepair method for a distributed mechanical system. IEEE Transaction on Robotics and Automation 15, 1035–1045 (1999) 29. Yim, M., Duff, D., Rufas, K.: PolyBot: A modular reconfigurable robot. In: IEEE Intl. conf. on Robotics and Automation, vol. 1, pp. 514–520 (2000) 30. Yim, M., Shen, W.M., Salemi, B., Rus, D., Moll, M., Lipson, H., Klavins, E., Chirikjian, G.S.: Modular self-reconfigurable robot systems - Challenges and opportunities for the future. IEEE Robotics and Automation Magazine 14(1), 43–52 (2007) 31. Zykov, V., Mytilinaios, E., Adams, B., Lipson, H.: Self-reproducing machines. Nature 435(7038), 163–164 (2005)

Analysis of Multi-robot Play Effectiveness and of Distributed Incidental Play Recognition Colin McMillen and Manuela Veloso

Abstract. Distributed play-based approaches have been proposed as an effective means of switching strategies during the course of timed, zero-sum games, such as robot soccer. However, play-based approaches have not yet been rigorously evaluated in a full robot soccer scenario. In this paper, we perform an extensive empirical analysis of play effectiveness with teams of robots. We show that different plays have a significant effect on opponent performance in real robot soccer games. In our analysis we further realized the problem of distributed play recognition: classifying the strategy being played by the opponent team. Play recognition in real robot soccer is a particularly challenging problem because our observations are only “incidental” — that is, the primary task of our team is to play soccer, not to explicitly observe members of the other team. Despite these challenges, we achieve high classification accuracy in the robot soccer domain. To achieve high accuracy, our team maintains a history of joint observations including team positions, opponent positions, and ball positions and utilizes hidden Markov models to recognize opponent plays.

1 Introduction Distributed robotic systems that operate in complex, uncertain environments may need to be equipped with multiple alternate approaches to solving the same tasks. For instance, a team that is traveling together through a crowded hallway may need to adopt a significantly different team-level behavior than a team that is navigating through a wide open space. In the past, plays have been successfully introduced as a means for a distributed team to adapt its high-level strategy to the features found in the environment [3]. However, plays have not yet been rigorously evaluated in a full robot soccer scenario. In this paper, we present an experimental study showing that Colin McMillen and Manuela Veloso Computer Science Department, Carnegie Mellon University, Pittsburgh, PA, U.S.A. e-mail: mcmillen,[email protected]

284

C. McMillen and M. Veloso

different plays have statistically significant effects on team performance in a robot soccer domain. We also consider the problem of distributed play recognition: combining our team’s observations to classify the behavior of the opponent team. In a real robot soccer game, our team members must focus their limited field of view on the most important objects in the environment, such as the ball and the localization landmarks. In contrast to a team that is fully engaged in tracking multiple moving objects, such as in [6], our team does not have the luxury of purposefully tracking opponent robots or spreading out to maximize the portion of the field that is viewed by the team. Despite these challenges, we show that our distributed team is able to effectively recognize the behavior of the opponent team, by maintaining a history joint observations. These joint observations include teammates’ own reported positions, positions of opponents observed in the environment, and the position of the ball as reported by each opponent. We utilize hidden Markov models to successfully classify the play used by the opponent team.

2 Background In previous work, we have presented a distributed, play-based strategy selection algorithm that allows a team of robots to autonomously change team strategies based on features such as the score of the game and the time remaining [3]. A play is a team plan that provides a set of roles; these roles are assigned to robots upon initiation of the play [1]. Each role specifies the top-level behavior of the associated robot. In the RoboCup domain, our robots’ roles are region-based: each robot is assigned to play in a specific region of the field, and will only pursue the ball when the ball is within its own region. A play selector runs continuously on one robot that is arbitrarily chosen to be the leader. As input, the play selector receives the state of the world (including the score of the game and the time remaining) and a playbook — the list of all strategies available to the team. The play selector finds the best play from the playbook and broadcasts the results to all teammates. In the past, we hand-coded a set of weights — one for each play in the playbook — and the play selector would choose the highest-weight play that was applicable in the current state. This allowed us to manually specify team behaviors such as “if there are fewer than two minutes remaining in the game and we are losing, play aggressively.” This algorithm was originally used in the RoboCup 2005 competition and was the first demonstrated instance of a distributed RoboCup team changing strategies in response to the score and time of the soccer game. One weakness of our previous work is that the play weights were hand-coded by humans, rather than selected optimally. We therefore introduced thresholdedrewards MDPs, which allow an agent to act optimally in domains in which the goal is to achieve a specified reward level within a given time deadline [4, 5]. In RoboCup, we treat each of our goals as +1 reward and each opponent goal as −1 reward; then the goal is to maximize the probability of achieving a positive cumulative reward by the end of the game. The solution to a thresholded-rewards MDP gives us the policy that optimizes our chances of winning in such a situation. In

Analysis of Multi-robot Play Effectiveness

285

previous work, we have shown that switching strategies based on score and time allows us to win more often than we lose, compared to an opponent that does not switch strategies — even if the opponent has strictly better low-level skills (i.e. probabilities of scoring at each time step) than our team does. However, this work only addressed a simple MDP-based model of the RoboCup domain, and was not actually implemented on real robots. In this paper, we address the problem of strategy switching in a real distributed robot team. We address two separate aspects of this problem: measuring the effectiveness of switching our own plays, and successfully recognizing the strategy played by the opponent team. All our results are based on extensive experiments using real robot teams in a realistic robot soccer domain.

3 Experimental Domain All the experiments presented in this paper are carried out in a domain based on the official RoboCup Four-Legged League rules for 2008 [8]. In each experiment, two teams of three robots play a 10-minute-long game of soccer. Both teams are from Carnegie Mellon’s recent CMDash’08 entry in the 2008 RoboCup US Open, which is an improvement of the CMDash’07 team that took third place (out of 24 teams) in the 2007 world competition [2]. Therefore, the underlying behaviors and skills of each team were identical; the only difference between experimental trials was the selection of plays for each team. For our experimental trials, we made the following changes to the official rules: • Each team consisted of three field player robots (rather than four) and no goalkeepers. • Due to the lack of goalkeepers, the illegal defender rule (which prevents the defending team’s field players from entering the goal box) was not enforced. • All other penalties, such as player pushing, leaving the field, and ball holding, were enforced. In case of a penalty, the offending player(s) were picked up and moved immediately to the halfway line (a “0-second standard removal penalty”) rather than remaining out of play for 30 seconds as specified in the rules. These changes were made such that a single human referee could successfully judge a game. In this paper, we focus on two separate plays: ROBO C UP and S UPER D EFENSE. The ROBO C UP play is the default play we have generally used in previous competitions [9]. This play assigns three roles: • an attacker robot that chases the ball over the entire field; • a defender robot that protects the defensive area of the field; and • a supporter robot that stays in the offensive region of the field. The ROBO C UP play is a balanced strategy that allows our team to score effectively and also provides a reasonable defense. In contrast, the S UPER D EFENSE play assigns all three field players to defensive roles:

286

C. McMillen and M. Veloso

(a) ROBO C UP play.

(b) S UPER D EFENSE play.

Fig. 1 Regions covered by each role in each play. (a) ROBO C UP play. The defender’s region is colored with dark dots; the supporter’s region is colored with diagonal lines. The attacker’s region is the entire field. (b) S UPER D EFENSE play. The rear defender’s region is colored with dark dots; the middle defender’s region is colored with a light checkerboard pattern, and the front defender’s region is colored with diagonal lines

• a front defender robot that covers approximately the same region as a normal defender robot, but stands much further forward when the ball is not in the defensive half of the field; • a middle defender robot that covers a slightly smaller region and usually stands about a meter behind the front defender; and • a rear defender robot that covers the area closest to the goal and usually stands about a meter behind the middle defender. Figure 1 shows the regions covered by each of the roles in the two plays. Typically, when playing S UPER D EFENSE, the front defender will immediately engage the ball when it enters the defensive half of the field. If the front defender fails to clear the ball, the ball will eventually enter the middle defender’s region; the middle defender will then join the front defender in trying to clear the ball. If the attacking team is still able to advance the ball forward near the goal, the rear defender joins in the defense and also attempts to clear the ball. By employing this strategy of defense-in-depth, the S UPER D EFENSE play is intended to make it very difficult for an opponent to successfully score a goal. However, the S UPER D EFENSE play comes with a significant drawback; namely, it is very unlikely that this play will ever score a goal. In a real game situation, we would never run the S UPER D EFENSE play for an entire game; rather, this is a strategy we would like to employ when our team is ahead near the end of the game and we wish to prevent the opponents from scoring an equalizer goal.

4 Multi-robot Play Effectiveness In this section, we aim to answer the question: do different team strategies produce significantly different outcomes when run against an opponent team? To test this,

Analysis of Multi-robot Play Effectiveness

(a) Opponent goals scored per game.

287

(b) Time between opponent goals.

Fig. 2 (a) Histogram of the number of goals scored per game for the two conditions. When our team uses the S UPER D EFENSE play, the opponents score significantly fewer goals. (b) Histogram of time between opponent goals. When our team uses the S UPER D EFENSE play, the opponents take significantly longer to score goals

we played a series of games against a static team configured to use the ROBO C UP strategy. For the control case, our team also played the ROBO C UP strategy. For the experimental case, our team played the S UPER D EFENSE strategy. We are interested in seeing if the S UPER D EFENSE play leads to fewer opponent goals than ROBO C UP, and also if the time between opponent goals is significantly higher for S UPER D E FENSE . Each case was tested in 12 independent 10-minute games, for a total of 4 hours’ worth of game time. To ensure fairness between the trials, the colors of the robots’ uniforms and the sides of the field were swapped between each trial. The final score of each game and the time of each goal was recorded. When playing ROBO C UP against ROBO C UP, our team scored 35 goals in total; this is a mean of 2.93 goals per game. The opponent team scored 42 goals in total, a mean of 3.50 goals per game. When playing S UPER D EFENSE against ROBO C UP, our team scored no goals; the opponent team scored a total of 22 goals, a mean of 1.83 goals per game. Figure 2(a) shows a histogram of the number of goals scored by our opponents in each of the games. We tested the statistical significance of the difference between the number of opponent goals scored per game using Student’s two-tailed t-test (assuming unequal variances) and a one-way analysis of variance (ANOVA). Both tests indicate that the number of opponent goals scored when we play S UPER D EFENSE is significantly fewer than the number of opponent goals scored when we play ROBO C UP (for the t-test, p = 0.0107; for the ANOVA, F(1, 23) = 7.8571, p = 0.0104). In addition to the number of goals scored by our opponents, we are also interested in how long it takes for the opponents to score. We therefore calculated the time between consecutive opponent goals in each of the games. At the end of the game, we pessimistically assume that the opponents score immediately after the game is over, since we don’t know how long it would have taken for the opponents to score their next goal (unless the game time remaining at the time of the last opponent goal was

288

C. McMillen and M. Veloso

less than 29 seconds, which was the fastest time between goals in any trial). This leads to a mean of 133.3 seconds between goals for the ROBO C UP condition, and a mean of 223.4 seconds between opponent goals for the S UPER D EFENSE condition. Figure 2(b) shows a histogram of the time between opponent goals for the two conditions. We again tested the statistical significance of the difference between the distributions using Student’s two-tailed t-test (assuming unequal variances) and a oneway ANOVA. Both tests indicate that the amount of time between opponent goals when we play S UPER D EFENSE is significantly greater than the amount of time between opponent goals when we play ROBO C UP (for the t-test, p = 0.0077; for the ANOVA, F(1, 85) = 10.7345, p = 0.0015). Taken together, these results show that the S UPER D EFENSE play successfully hinders the opponent team from scoring. Compared to ROBO C UP, the S UPER D E FENSE play significantly reduces the number of goals scored by the opponent and also increases the amount of time between opponent goals. However, the S UPER D E FENSE play completely prevents our own team from scoring. In order to maximize the probability of winning, our team should therefore choose the ROBO C UP play by default, switching to S UPER D EFENSE when ahead by a significant margin, in order to prevent the opponent from mounting a successful comeback.

5 Distributed Incidental Play Recognition In the previous section, we considered the effect of switching our own play in order to prevent the opponent team from scoring goals. In this section, we introduce the problem of distributed incidental play recognition — recognizing the behavior of the opponent team from our own team’s observations. This is an interesting distributed perception problem because our team’s observations of the environment and other robots are incidental rather than purposeful. By this, we mean that the primary task of our team is not to observe the environment and classify the behavior of the other team; rather, our task is to play soccer well, and any observations of the environment or opponent robots happen by “accident” while we play soccer. Since the AIBO camera’s field of view is very narrow — under 60 degrees — each robot has a very limited focus of attention, which is usually focused on the ball since the ball is the most important object in the environment. In particular, our team does not explicitly track the movements of the opponent robots nor attempt to maximize the portion of the field that is viewed. In this paper, we use hidden Markov models (HMMs) to model the behavior of the other team [7]. To enable distributed play recognition, each of our robots communicates the following data once per second: • The robot’s best estimate of its own position. • The robot’s best estimate of the position of the ball. • The position of the last opponent robot seen by the robot.

Analysis of Multi-robot Play Effectiveness

289

(a) RoboCup, first minute.

(b) SuperDefense, first minute.

(c) RoboCup, full game.

(d) SuperDefense, full game.

Fig. 3 Sample log data from two games. Lines indicate the positions of our own robots, circles indicate observations of the ball, and squares indicate observations of opponent robots. Our goal is located on the right side of each figure; the opponents’ goal is located on the left side of each figure

All positions are communicated in global coordinates, so the team has a common frame of reference. For each of these features, each robot also broadcasts a boolean feature indicating whether the observation is considered “valid”. For example, the robot’s estimate of its own position is valid iff the robot’s estimate of localization error is relatively low. For purposes of this experiment, the data broadcast by each robot was also sent to an offboard computer for later processing; in a real game situation the robots would only broadcast to each other and use the results to classify the opponent behavior online. Figure 3 shows some sample data logged by the offboard computer from a typical ROBO C UP game and a typical S UPER D EFENSE game. Figure 3(a) shows data from the first minute of a game in which the opponents are playing the ROBO C UP strategy; Figure 3(b) shows data from the first minute of a game in which the opponents are playing the S UPER D EFENSE strategy. Figures 3(c) and 3(d) respectively show the data from entire ROBO C UP and S UPER D EFENSE games. Even though the opposing S UPER D EFENSE team does not penetrate significantly into the offensive half of the field, there is still a significant overlap in observations between the ROBO C UP and S UPER D EFENSE games. These

290

C. McMillen and M. Veloso

Algorithm 1. P LAY-R ECOGNITION algorithm 1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14:

Given: training sets RCtrain and SDtrain , observation sequence J of length t λRC ← BAUM -W ELCH(RCtrain) λSD ← BAUM -W ELCH(SDtrain) for i = 1 to t do Ji ←  j1 , j2 , . . . , ji  pRC ← P(Ji |λRC ) pSD ← P(Ji |λSD ) if pRC ≥ pSD then C[i] ← ROBO C UP else C[i] ← S UPER D EFENSE end if end for return C[i]

overlapping observation histories show that correct classification of opponent plays is not a trivial task in the robot soccer domain. Formally, the observation of a single robot r at timestep t is: or,t = bxr,t , byr,t , bvr,t , pxr,t , pyr,t , pvr,t , oxr,t , oyr,t , ovr,t ,

(1)

where bxr,t and byr,t give the position of the ball in global coordinates as seen by robot r at time t and bvr,t is a boolean flag saying whether robot r considers the ball observation to be valid. Similarly, pxr,t , pyr,t , and pvr,t give the position and validity of robot r’s own position and oxr,t , oyr,t , and ovr,t give the position and validity of the opponent most recently observed by robot r. We then define the joint observation jt of the team at time t as the combination of the individual robots’ observations at time t: jt = o1,t , o2,t , o3,t . Over the course of an entire game that is t timesteps long, the robots obtain a joint observation sequence J =  j1 , j2 , . . . , jt . Algorithm 1 presents our play-recognition algorithm. This algorithm assumes we have a training set RCtrain consisting of a set {J1 , J2 , . . . , Jn } of observation sequences gathered while the opponent was playing the ROBO C UP play, and another training set SDtrain consisting of a set {J1 , J2 , . . . , Jn } of observation sequences gathered while the opponent was playing the S UPER D EFENSE play. On lines 2–3 of the play-recognition algorithm, we train one HMM λRC by providing the training sequences RCtrain as input to the Baum-Welch algorithm; we also train another HMM λSD by providing the training sequences SDtrain as input to the Baum-Welch algorithm. Lines 4–13 provide online classification at each timestep i. Ji is the vector of all joint observations  j1 , j2 , . . . , jt  seen by the team in the first i timesteps. On lines 6–7, we compute the likelihood of Ji according to the two models λRC and λSD . These likelihoods are computed using the forward algorithm. The model λ which maximizes the likelihood of the observation sequence is chosen as the best estimate of which play the opponent team is running. The classification output is stored in the array C.

Analysis of Multi-robot Play Effectiveness

291

Algorithm 2. Cross-validation procedure 1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13:

Given: sets of observation sequences RC and SD, each of size k correct ← 0; incorrect ← 0 for i = 1 to k do rc ← RCi sd ← SDi Crc ← P LAY-R ECOGNITION(RC− {rc}, SD − {sd}, rc) Csd ← P LAY-R ECOGNITION(RC− {rc}, SD − {sd}, sd) correct ← correct + number of elements e in Crc s.t. e = ROBO C UP incorrect ← incorrect + number of elements e in Crc s.t. e = ROBO C UP correct ← correct + number of elements e in Csd s.t. e = S UPER D EFENSE incorrect ← incorrect + number of elements e in Csd s.t. e = S UPER D EFENSE end for return correct / ( correct + incorrect )

To collect training data, we ran twelve 10-minute games against an opponent team running the ROBO C UP play and twelve 10-minute games against an opponent team running the S UPER D EFENSE play. In all trials, our own team was running the ROBO C UP play. Let RC be the set consisting of the 12 joint observation sequences gathered when playing against a ROBO C UP opponent; let SD be the set consisting of the 12 joint observation sequences gathered when playing against a S UPER D EFENSE opponent. We perform leave-one-out cross-validation to evaluate the effectiveness of our approach. Algorithm 2 presents our cross-validation procedure. Since we have 12 observation sequences for each case, the main loop (lines 3–12) runs 12 times. In each iteration, one element sd is held out of SD and one element rc is held out of RC. The remaining observation sequences are used to classify sd and rc, and the number of successful and unsuccessful classifications is recorded. The procedure returns the fraction of timesteps which were successfully classified. In order to achieve good classification performance, we need a good domain representation. We are therefore interested in selecting an informative set of features from the raw joint observations. In principle, we can apply any arbitrary function f ( j) to map a single joint observation to a vector of features. In this paper, we only consider feature-selection functions of a limited form. Namely, f ( j) can only perform two operations: 1. Filtering out some features, removing them from the joint observation entirely. Some of the fields communicated by the robots may not be useful for play recognition, in which case the models may overfit if these fields are present. Filtering allows useless features to be ignored. 2. Re-ordering features in the joint observation. It may be more useful for a specific field in the feature vector to represent “the position of the robot closest to the opponent goal” rather than “the position of robot 1”. Feature re-ordering allows such semantics to be an explicit part of the model.

292

C. McMillen and M. Veloso

To help understand which features are most important for successful play recognition in robot soccer, we tried a variety of feature-selection functions and computed the overall classification accuracy of each. By itself, ball position is the most informative feature. If the observation vectors include only the ball’s x- and y-coordinates plus the “ball valid” flag, and the ball observations are sorted by their x-coordinates, we achieve an overall accuracy of 86.98%. In our coordinate system, the center of the field is (0, 0), and positive x points towards our own goal, so sorting in ascending order corresponds to ordering the ball observations from “nearest the opponent goal” to “furthest from the opponent goal”. Using only the reported x-positions of our own robots (sorted by x) achieves an accuracy of 82.34%. It is interesting to note that adding our own y-positions actually lowers our classification accuracy to 76.65%, due to overfitting. Adding the “own position valid” flag also hurts classification accuracy; we assume that this is because the robots’ position confidence estimates were tuned for single-robot behavior, such as deciding when the robot needs to look at localization landmarks, rather than team behavior or opponent recognition. By itself, using the x-positions of opponent robots (sorted, plus the “opponent valid” flag) performs quite poorly, achieving an accuracy of only 69.54%. Adding the y-positions of opponents also decreases classification accuracy, to 63.74%, which we also attribute to overfitting. If the feature vector incorporates the ball position plus our own positions, overall accuracy increases to 87.44%. If we also include the opponent positions, we reach our best accuracy, of 88.63%. We conclude that the position of the ball is by far the most important feature; however, including additional information can help somewhat in classification performance. This is unsurprising given the incidental nature of our classification task — since most of the robots on our team spend most of their time focusing their attention on the ball, the ball is the object that is seen most and has the lowest observation error. The accuracy levels achieved with different sets of features are summarized in Figure 4. Through the selection of the proper set of features, we can achieve classification accuracy of 88.63%. One question remains: in which circumstances do we still tend to make errors? Figure 5 shows our classification accuracy at each time step of a typical ROBO C UP game and a typical S UPER D EFENSE game. We can see from the figure that all of our classification errors occur early in the game, before we have collected many observations. After about 100 seconds have elapsed, our classification Features used Classification accuracy  ball x, ball y, ball valid  86.98%  own x  82.34%  opp. x, opp. valid  69.54%  ball x, ball y, ball valid, own x  87.44%  ball x, ball y, ball valid, own x, opp. x, opp. valid  88.63% Fig. 4 Summary of classification accuracy when using different sets of features as input to the HMM

Analysis of Multi-robot Play Effectiveness

293

Fig. 5 Classification accuracy at each timestep for a typical ROBO C UP game and a typical S UPER D EFENSE game

accuracy for both games is 100%. In fact, this is the typical pattern seen in almost all the games: the classification accuracy is rather poor at the beginning, but improves significantly by the time 100–200 seconds have elapsed. We therefore claim that in a real robot soccer scenario, we would not want to change the behavior of our own team to adjust to the opponents until either a certain amount of time has elapsed or the likelihoods of the observation sequence given each of the models have diverged significantly.

6 Conclusion In this paper, we performed an extensive empirical analysis of the performance of two different plays, ROBO C UP and S UPER D EFENSE, in four hours of real robot soccer games. We have shown that our team’s play choice has a significant effect on the opposing team’s performance. Namely, when we choose the S UPER D EFENSE play, the opponents score fewer goals per game, and take longer on average to score each goal, than when we choose the ROBO C UP play. Both these results are statistically significant. These results indicate that the optimal play-selection strategy for our team — the policy that maximizes the probability of winning — is to play the ROBO C UP play by default and to switch to the S UPER D EFENSE play when our team has a significant lead. Switching to S UPER D EFENSE decreases the likelihood that the opponent team can make a comeback. In this paper, we also introduced the problem of distributed incidental play recognition, in which a distributed team of robots attempts to recognize the behavior of an opponent team while engaged in a primary task (such as robot soccer) that is not

294

C. McMillen and M. Veloso

optimized for behavior recognition. Our team members periodically communicate their observations of their own position, the position of the ball, and the position of opponent robots. Using an approach based on hidden Markov models, our team can accurately classify the behavior of the opponent team 88.63% of the time. Many of the classification errors are made at the very beginning of each game, when there is less data available to the classifier. In the future, we look forward to further integrating optimal play switching and opponent play recognition into our full RoboCup soccer team. Acknowledgements. The authors would like to thank Douglas Vail for providing us with the HMM software that we used for play recognition. This research was sponsored in part by United States Department of the Interior under Grant No. NBCH-1040007, and in part by the Boeing Corporation. The views and conclusions contained in this document are those of the authors only.

References 1. Bowling, M., Browning, B., Veloso, M.: Plays as team plans for coordination and adaptation. In: Proceedings of the 14th International Conference on Automated Planning and Scheduling (ICAPS 2004) (2004) 2. Fasola, J., Veloso, M.: Real-time object detection using segmented and grayscale images. In: Proceedings of the 2006 IEEE International Conference on Robotics and Automation (ICRA 2006) (2006) 3. McMillen, C., Veloso, M.: Distributed, play-based role assignment for robot teams in dynamic environments. In: Proceedings of Distributed Autonomous Robotic Systems (2006) 4. McMillen, C., Veloso, M.: Thresholded rewards: Acting optimally in timed, zero-sum games. In: Proceedings of the Twenty-Second Conference on Artificial Intelligence (AAAI 2007) (2007) 5. McMillen, C., Veloso, M.: Unknown rewards in finite-horizon domains. In: Proceedings of the Twenty-Third Conference on Artificial Intelligence(AAAI 2008) (2008) 6. Parker, L.E., Emmons, B.: Cooperative multi-robot observation of multiple moving targets. In: Proceedings of 1997 International Conference on Robotics and Automation (1997) 7. Rabiner, L.R.: A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE 77(2), 257–286 (1989) 8. RoboCup Technical Committee: Robocup four-legged league rule book (2008), https://www.tzi.de/4legged/pub/Website/Downloads/ AiboRules2008.pdf 9. Vail, D., Veloso, M.: Dynamic multi-robot coordination. In: Multi-Robot Systems: From Swarms to Intelligent Automata, vol. II, pp. 87–100. Kluwer Academic Publishers, Dordrecht (2003)

Sparsing of Information Matrix for Practical Application of SLAM for Autonomous Robots Sparsing of of Information Matrix for Practical Application SLAM 1 Haiwei Dong and Zhiwei Luo H. Dong and Z. Luo

Abstract. As simultaneously localization and mapping (SLAM) is the basis for autonomous robots to realize high level task, it has become a key issue in mobile robotics field to propose a practical SLAM approach for a large scale environment. By analyzing the features of information matrix, this paper presents a novel method to enhance practicability of simultaneous localization and mapping (SLAM) by changing information matrix into a sparse matrix. A large scale environment simulation shows that our sparsification method is highly efficient with the increase of landmarks while maintaining accuracy. Outdoor experiment verifies the promising future of our approach for the application into the real-world.

1 Introduction In unknown environment, mobile robot determines its own position from the map while mapping the environment according to its own localization is called simultaneous localization and mapping (SLAM). As SLAM is the basis for an autonomous robot to realize high level task, e.g. rescue in large environment, many researchers focus on solving SLAM in real world. In the past decades, SLAM has gained great progress and is applied in various environments, from classical land case [1] to deep sea case [2]. By now, the development of SLAM can be divided into three main stages. The first stage concerns about the model of SLAM problem. Introducing uncertainty into motion model and observation model is really novel work. In the second stage, researchers begin to solve SLAM problem from the view of probabilistic process. The milestone of stage two is Smith et al.’s research which solves SLAM by using extended Kalman filter (EKF) for the first time [3]. However, the computational burden of EKF-SLAM requires up to O (n 2 ) , where n denotes the number of mapped landmarks. In other words, EKF-SLAM can not be applied in real-world for its heavy computational burden. From then on, more and more researchers Haiwei Dong Kobe University, Kobe, Japan e-mail: [email protected] Zhiwei Luo Kobe University, Kobe, Japan e-mail: [email protected]

296

H. Dong and Z. Luo

address the problem of enhancing SLAM algorithm’s efficiency. So it comes to the third stage when various kinds of SLAM methods come out. The main direction of this stage is to propose efficient SLAM approach, such as scan matching [4, 5], decoupled SLAM [6], Fast SLAM [7], EIF-SLAM [8] etc. This paper presents a novel approach to solve SLAM problem based on sparsification of information matrix. By analyzing the three dimensional structure of information matrix, three main features of information matrix are pointed out. Then we propose a sparsification algorithm to wipe out nearly half of the elements in information matrix. After that, the information matrix is a real sparse matrix. By applying special algebraic operation rules of sparse information matrix, our solution of SLAM can gain high efficiency. Finally, our new approach is simulated in a large scale (400 landmarks) complicated environment. The simulation results not only show that our method is much more efficient compared with EIF-SLAM but also explain the mechanism of our approach. Experiment verifies the promising practicability into real-world.

2 Fundamental Idea of EIF-SLAM Definition of environment variables (EV) and matrix block of information domain variables (IDE) are shown in Table 1. Table 1 Definition of EV & IDE

Matrix block of information estimate variables

Environment variables

x

Robot state

ηx

Information vector of x

M

Landmark state

ηM

Information vector of M

Information matrix block (r. vs. r.) Λ xM Information matrix block Observation set Z (r. vs. m.) Λ Mx Information matrix block Control signal set M (m. vs. r.) Λ MM Information matrix block (m. vs. m.) ( r. denotes robot, m. denotes mapped landmark, vs. means versus)

ε

Environment state

Λ xx

SLAM problem is formulated as to estimate environment state from motion control signals and observations. The main steps of EIF-SLAM includes motion update, add features and observation update which are marked by star in Fig.1.

Sparsing of Information Matrix for Practical Application of SLAM

297

In every computational circle, robot moves according to the motion control signals meanwhile updating information domain invariants. Then the robot observes the whole environment from the new location and recognizes landmarks from the environment as well. After that, recognize landmark pairs which reflect the corresponding relation between environmental landmark and mapped landmark. If the landmark has been mapped, update the information domain variants by observations. Otherwise, add the landmark state into ε [8].

Fig. 1 Block diagram of EIF-SLAM algorithm

From probabilistic view, SLAM problem can be stated as the posterior probability of ε under the known condition of U and Z . In addition, previous research concludes that SLAM estimation satisfies [9]

p(ε | Z ,U ) ∝ e

1 − ε T Λε +ηε 2

⎛ ⎡η ⎤ ⎡ Λ = N −1 ⎜ ⎢ x ⎥ , ⎢ xx ⎝ ⎣η M ⎦ ⎣ Λ Mx

Λ xM ⎤ ⎞ ⎟ Λ MM ⎥⎦ ⎠

(1)

298

H. Dong and Z. Luo

Equation (1) shows that the distribution of ε satisfies the conditions which EIF requires. So it’s reasonable to apply EIF in SLAM.

3 Sparse Information Matrix Directly 3.1 Features of Information Matrix Three dimensional portraits of information matrix are shown in Fig. 2.

800

4

600 400

2

value

value

200 0

0

−200

−2

−400 −600 100

80

80

100 40

80

60

60 20 column index

60

40

50

40 20

0

0

row index

column index

(a) global view

20 row index

(b) local view

Fig. 2 Three dimensional portraits of information matrix

where (a) shows the global view of information matrix. And (b) displays the local part of information matrix which masks neighborhood of main-diagonal line and endpoint of sub-diagonal line as well. In fig.2 row and column denote the position index of one element in information matrix. Value shows the numerical value of the corresponding element, i.e. the value of element (row index, column index). From fig.2, it is reasonable to divide the information matrix into two parts. Part one is the neighborhood of main diagonal line and sub-diagonal endpoint. And Part two is the other part of information matrix. So it’s simple to conclude that information matrix has three main features: a) b) c)

Information matrix is symmetric along the main diagonal line; Elements with very huge values concentrate in part one. For the elements of part two, further the element away from the main diagonal line, smaller the value of the element is. The detail explanation of the features is shown in Table 2. The size of information matrix balloons when the scale of environment gets larger. In addition, the main computational burden concentrates on calculation related to information matrix. Therefore, it’s very necessary to break some weak

Sparsing of Information Matrix for Practical Application of SLAM

299

Table 2 Analysis of information matrix’s features

Feature

By EIF-SLAM formula

a)

Information matrix is a positive symmetric Hermit matrix [9].

By Bayes Network

b)

From the view of Bayes Network, the elements in diagonal line denote the link strength of robot versus itself, or landmark versus itself; the elements near the endpoint of information matrix show the link strength of robot versus current observing landmark [8].

c)

From graph theory, further away from the main diagonal line the element is, stronger it denotes the link is [8].

links, i.e. wipe out some elements with small values in information matrix, to enhance efficiency of information matrix. Because elements in part one is very essential to maintain the robustness of the algorithm, it’s reasonable to sparse information matrix by dealing with the elements in part two.

3.2 Sparsing Information Matrix Directly It’s obvious that sparser the information matrix is, more efficient the algorithm becomes. However, the information which denotes link information looses while sparsification. Inconsiderable sparsification may lead the algorithm ruined. Therefore we must find a compromise of efficiency and accuracy. According to the main features and value distribution of information matrix, a new sparse method is proposed below. Algorithm 1. Sparsification 1) 2)

3)

Set parameter λ , N and initialize rindex , cindex , temp = dim( Λ k ) Applicable check: z if k > N do procedure 3) z otherwise go to procedure 4) Sparsification: compute threshold = 1 / λ dim(Λ k )2 Λ −k 1−1 for rindex from 1 to dim( Λ k )

300

H. Dong and Z. Luo

for cindex from 1 to temp z

if Λ(rindex , cindex ) ≤ threshold , set a) b)

Λ ( rindex , cindex ) = 0 , Λ (cindex , rindex ) = 0 temp = temp − 1

z

4)

otherwise break present circle endfor endfor End the algorithm

Where 0 < λ ≤ 1 is conservative coefficient, N is initial step, Λ k is the information matrix at time k , dim( Λ k ) is the dimension of Λ k . In fact, only motion update procedure causes information matrix dense, so it is reasonable to sparse information matrix once in every computing cycle. After sparsification, information matrix becomes exactly sparse. However, sparsification error comes out. In practical application, the error coming from sparsification decreases apparently by loop-closure (i.e. the trajectory of robot’s movement becomes a circle). In fact, loop-closure reduces sparsification error by correcting the landmark pairs which has the largest error. Suppose Λ and Λ + E are the information matrix before and after sparsification, where E is the error coming from sparsification. From lots of simulation under various situations, we have the following conclusions. 1) When E Λ −1 < 1 , estimation result remains accurate; 2) When E Λ −1 >> 1 , sparsification error becomes large. It is obvious that our sparsification algorithm satisfies E Λ −1 < 1 . Therefore, algorithm 1 maintains accuracy of esimation while changing information matrix exactly sparse.

4 Simulation The new method is simulated in a large scale environment (about 400 landmarks). As the landmarks are very dense, the environment is a complicated. The robot moves clockwise for two laps while utilizing SLAM estimation. The robot applied in simulation is a two wheel robot with 3 degree of freedom and the sensor arranged on the robot is range-bearing type sensor (such as laser, camera, etc.). The simulation parameters are set as followed: the maximum range of laser scan is 12m; the speed of the robot is of 1m/s; control time circle is 50ms.

Efficiency As shown before, the main computational burdens of SLAM estimation come from three procedures: motion update, adding features and observation update.

Sparsing of Information Matrix for Practical Application of SLAM Fig. 3 Efficiency comparison

add features

motion update

EIF−SLAM

observation update

our method

1.5 1 0.5 0

0

50

100

150

200

250

300

350

400

0

50

100

150

200

250

300

350

400

0

50

100 150 200 250 300 current size of environment state

350

400

1.5 1 0.5 0

error

301

1.5 1 0.5 0

convariance

x

10 0 −10

0

50

100

150

200

250

300

350

400

450

500

0

50

100

150

200

250 Time

300

350

400

450

500

y

20 0 −20

(a) error

convariance

x

10

0

−10

0

50

100

150

200

250

300

350

400

450

500

0

50

100

150

200

250 Time

300

350

400

450

500

y

20

0

−20

(b) Fig. 4 Comparison of error and covariance

The efficiency comparison of EIF-SLAM (denoted as star) and our method (denoted as cross) is shown in Fig.3. It is obvious that our method is much more efficient. When the scale of environment gets larger, our method gains more advantages in efficiency.

302

H. Dong and Z. Luo

Decrease of sparsification error Considering of error and covariance during SLAM estimation procedure, we can gain insight into the functionality of loop-closure. The comparison of error and covariance is shown in Fig.4. Where dash line and solid line denote estimation error and covariance of robot’s position, horizontal-axis represents time. Estimation error and covariance decrease rapidly (almost jump) at time 250 which indicates that there is a loopclosure at time 250. Compared with EIF-SLAM, the estimation error and covariance of improved algorithm is a little larger in the first lap (Fig.4(a)). But in the second lap, the estimation gap between EIF-SLAM and our method seems small (Fig.4(b)). In other words, by using loop-closure properly, our algorithm can reach high accuracy while running much efficiently.

5 Experiment Car park is a famous benchmark in SLAM research field. We utilize this dataset to verify our algorithm. In this experiment, the environment is an outdoor car park. And the robot is a four-wheel car on which GPS, laser are mounted. In fact, we can get the map of the environment as well as the position of the robot with GPS data. However, GPS data may not be gotten in some real word cases, such as the city with many high builders, the cave in the ground and so on. In these cases, we have to estimate the position of robot and landmarks by SLAM. In a word, the main aim of this experiment just to verify our SLAM algorithm. And the GPS data are being used only in the evaluation of accuracy. In the experiment, we recognize cylindrical object as landmarks and use Individual compatibility nearest neighbor (ICNN) in data association. Because the landmarks are not very dense, the mismatching of landmarks would not occur. Furthermore, the trajectory of the robot consists of many circles, that is to say, loop-closure occurs several times. According to the previous analysis in the simulation, loop-closure could decrease the estimation error effectively. Hence, the experiment also proves this analysis by real-world data. Experiment results are shown in Fig.5 and Fig.6. where in Fig.5(a) circle denotes real position of robot which is measured by GPS, and in Fig.5(b) triangular and cross denote the estimation positions of robot and landmark respectively, eclipse represents 3 σ uncertainty. Fig.6 records the localization error of robot where x-axis represents time; y-axis shows the estimation errors of robot. Upper subplot shows the error in latitude direction while lower subplot shows that in longitude direction, respectively. Because the dataset doesn’t contain the information relating to the robot’s direction, the real position of landmarks can’t be determined. But as the original idea of SLAM algorithm is to consider the position of robot and landmarks together, the estimation accuracy of robot and landmarks is consistent. That is to say, if the position of robot is estimated accurately (Fig.6), the estimation of landmarks should also be equivalently accurate.

Sparsing of Information Matrix for Practical Application of SLAM

303

Longtitude (m)

15

10

5

0

−5 −25

−20

−15

−10

−5 0 Lantitude (m)

5

10

15

(a)

Longitude (m)

15

10

5

0

−5 −25

−20

−15

−10

−5 0 Latitude (m)

5

10

15

(b) Fig. 5 Experiment results Fig. 6 Estimation error of robot Latitude (m)

4

2

0

−2

200

400

600

800

1000

200

400

600 time

800

1000

Longitude (m)

2 0 −2 −4 −6

304

H. Dong and Z. Luo

By utilizing our sparsification algorithm, nearly half of the elements in information matrix are set to zeroes. As many calculations relating to zero is so easy that dealing with such information matrix is really out of problem. Further more, by choosing different value for parameter λ and N , it can be determined whether the SLAM algorithm is more accurate or more efficient considering of the characteristics of the environment.

6 Conclusion Computational efficiency and practicability has become quite crucial while SLAM technology becomes much more mature. In this paper, according to the features of information matrix, a new approach for SLAM is introduced. Owning to the high efficiency of the special rules for sparse operation, our method solves SLAM efficiently by sparsing information matrix directly. The errors coming from sparsification decrease apparently by loop-closure. In other words, the new algorithm realizes efficiency with accurate estimate. The results of simulation in large scale environment as well as outdoor experiments testify the validity of our method.

References 1. Yufeng, L., Thrun, S.: Results for outdoor-SLAM using sparse extended information filters. In: IEEE International Conference on Robotics and automation, pp. 1227–1233 (2003) 2. Eustice, R., Singh, H., Leonard, J., Walter, M., Ballard, R.: Visually navigating the RMS Titanic with SLAM information filters. In: Robotics: Science and Systems, pp. 1223–1242 (2005) 3. Smith, R., Self, M., Cheeseman, P.: Estimating uncertain spatial relationships in robotics. Springer, NewYork (1990) 4. Bosse, M., Roberts, J.: Histogram matching and global initialization for laser-only SLAM in large unstructured environments. In: IEEE International Conference on Robotics and Automation, pp. 4820–4826 (2007) 5. Andreas, N., Kai, L., Joachim, H., Hartmut, S.: 6D SLAM - 3D mapping ourdoor environments. J. Field Robotics 24, 699–722 (2007) 6. Zhan, W., Shoudong, H., Gamini, D.: D-SLAM: a decoupled solution to simultaneous localization and mapping. J. Robotics Research 26, 187–204 (2007) 7. Chanki, K., Sakthivel, R., Kyun, W.: Unsented Fast-SLAM: a robust algorithm for the simultaneous localization and mapping problem. In: IEEE International Conference on Robotics and Automation, pp. 2439–2445 (2007) 8. Eustice, M.R., Singh, H., Leonard, J.J.: Exactly sparse delayed-state filters. In: IEEE International Conference on Robotics and Automation, pp. 2417–2424 (2005) 9. Thrun, S., Koller, D., Ghahramani, Z., Durrant, W., Ng, A.: Simultaneous mapping and localization with sparse extended information filters: theory and initial results. Carnegie Mellon University (2002)

Trajectory Generation for Multiple Robots of a Car Transportation System Mitsuru Endo, Kenji Hirose, Yusuke Sugahara, Yasuhisa Hirata, Kazuhiro Kosuge, Takashi Kanbayashi, Mitsukazu Oomoto, Koki Suzuki, Kazunori Murakami, and Kenichi Nakamura

Abstract. We propose a car transportation system referred to as iCART (intelligent Cooperative Autonomous Robot Transporters). In this system, two robots approach the car autonomously, and lift up it for supporting its weight from right/left side respectively. Then, two robots transport the car in coordination. Collision-free trajectories for approaching and transporting the car are generated based on the potential field in the configuration space. In this paper, we describe details of the way to construct a configuration space, to apply a potential to the configuration space, and to generate a trajectory. Additionally, for generating the collision-free trajectory on the environment where two robots exist, repulsive potentials are applied to each robot. Described methods are integrated to iCART and experimental results illustrate the validity of the proposed method.

1 Introduction We propose a car transportation system using multiple robots referred to as -iCART(intelligent Cooperative Autonomous Robot Transporters) shown in Fig.1[1]. This system realizes the efficient transportation of the car at the narrow space and the situation a driver cannot or should not get into the car or ignite its engine, because this system handles the car freely unaffected by non-holonomic constraints of the car by using the omni-directional mobile mechanism and wheel lifting mechanisms of the robot. Skilled drivers for the narrow space, wreckers, surrogate drivers, etc are not Mitsuru Endo, Kenji Hirose, Yusuke Sugahara, Yasuhisa Hirata, and Kazuhiro Kosuge System Robotics Laboratory, Department of Bioengineering and Robotics, Tohoku University Takashi Kanbayashi, Mitsukazu Oomoto, Koki Suzuki, Kazunori Murakami, and Kenichi Nakamura Ishikawajima Transport Machinery Co., Ltd. e-mail: [email protected]

306

Fig. 1 Car Transportation System iCART

M. Endo et al.

Fig. 2 Car Transportation by Multiple Robots

required by using this system. We consider the transportation system is available for any kind of car transportations: the valet parking service, the tow away service, the parking, and the transportation of cars at the factory, the ferry, the parking area, etc. This system is composed of two robots and laser range finders attached on the ground. Two robots approach right/left side of the car respectively based on information of laser range finders, and lift up the car as shown in Fig.2. Then robots transport the car in coordination. In this paper, we describe the method to generate trajectories to approach the car for lifting up it, and to reach a destination for the transportation it. For avoiding the collision of robots, the field, obstacles and the car is discussed. The trajectory is generated using the Laplace potential method in the configuration space. We use the repulsive potential applied to robots for avoiding the collision of robots. By integrating these algorithms, the car transportation is realized.

2 Trajectory Generation of the Robot In this chapter, let us consider how to generate collision-free trajectories reaching the destination. In this study, to generate a collision-free trajectory, the potential method is used in the configuration space.

2.1 Configuration Space The three dimensional configuration space is used for generating the trajectory: two dimensions for the position of the robot, and one dimension for the orientation of the robot[5]. In the configuration space, the trajectory generation of the robot which have an omni-directional mobile mechanism, becomes easy and simple, because the orientation of the robot is treated one component of a vector. Additionally, because the configuration space includes size information of the robot measuring contacts with the field or obstacles, we can treat the robot as a point. In this paper, the field, robots, and obstacles are quantized and represented as grids in the configuration space. We use the Minkowski addition[3] to measure

Trajectory Generation for Multiple Robots

307

Fig. 3 Two Dimensional Configuration Space

Fig. 4 Three Dimensional Configuration Space

the contact between the robot and the field or obstacles. Equation (1) shows the Minkowski addition. A ⊕ B = {a + b|a ∈ A, b ∈ B} (1) where, A and B are a set respectively. Fig.3 shows the two dimensional configuration space generated using the Minkowski addition at each orientation of the robot, respectively. By accumulating two dimensional configuration spaces as layers, a three dimensional configuration space is generated as shown in Figure.4.

2.2 Laplace Potential Method The potential method is applied to the configuration space for generating the trajectory. The destination of the trajectory is given the minimum potential and the contact

308

M. Endo et al.

area is given the maximum potential. By defining the robot expressed as a point goes to the lower potential, the robot reach the destination contacting with nothing. In this study, for realizing the deadlock-free trajectory generation, the Laplace potential field is used[2]. By using the Laplace potential field, there are no local minimum value other than the destination given the minimum potential at first. Laplace potential filed U satisfy a following equation.

∂ 2 U ∂ 2U ∂ 2U + + =0 ∂ x2 ∂ y2 ∂ θ 2

(2)

where, U ∈ R3×3 is the Laplace potential and x, y, θ ∈ R are each component of the configuration space. Because the configuration space is quantized, the discrete Laplace potential equation[2] shown in (4) is used.   U = ui, j,k (0 ≤ i ≤ l, 0 ≤ j ≤ m, 0 ≤ k ≤ n) (3)  1 ui+1, j,k + ui−1, j,k + ui, j+1,k + ui, j−1,k + ui, j,k+1 + ui, j,k−1 (4) ui, j,k = 6 where, ui, j,k ∈ R is a component of the potential U and l, m, n ∈ R are the number of divides of x, y, θ , respectively.

2.3 Steepest Descent Method The steepest descent method is used for generating a path of the robot. Equation (5) describes the steepest descent method.   ∂ Uk ∂ Uk ∂ Uk pk+1 = pk − τ (5) ∂ xk ∂ yk ∂ θk where, p = [x, y, θ ]T is the position of the robot path, and index k ∈ R means the kth calculation. τ ∈ R is a constant parameter relating the calculate time. In this study, the potential field U is expressed as a discrete function. Thus, we derive the continuous form of the potential on the robot expressed as a point by (6). 1

U(x, y, θ ) =

1

1

∑ ∑ ∑ (−1)α +β +γ ui+α , j+β ,k+γ

α =0 β =0 γ =0



yi+α , j+β ,k+γ − y + 1 − β





xi+α , j+β ,k+γ − x + 1 − α

θi+α , j+β ,k+γ − θ + 1 − γ



 (6)

where, U(x, y, θ ) ∈ R is the potential on the robot located at (x, y, θ ), and ui, j,k is the component of the potential U on the nearest grid from the robot. Equation.(6) interpolates the difference of the potential between grids. To explain (6), the two dimensional case is shown in Fig.5. This figure shows a square region, including the point (x, y), formed by the center of surrounding four girds, is divided into four rectangles related in the position of the point. The potential on the point (x, y) is calculated by the summation of that on surrounding four grids weighted by

Trajectory Generation for Multiple Robots

309

Fig. 5 Two Dimensional Equation of Calculation of Potential in Continuous Form

the reciprocal ratio to the area of four rectangles. Equation.(6) is derived as the three dimensional case of Fig.5. To substitute U(x, y, z) derived by (6) to (5), the path is generated on the configuration space.

2.4 Trajectory Generation A collision-free trajectory is generated from the path generated by the method described on above section. The generated path is a sequence of points which have a constant pitch based on the constant τ in (5). For generating the continuous trajectory, the spline interpolation method is used. When all of points are interpolated by the spline function, the computational cost increases. Thus, in this paper, interpolating points are sampled at constant pitch from the sequence of points of the path for reducing the computational cost.

3 Collision-free Trajectory for Multiple Robots The trajectory generating method described in above chapter only considers collisions of the robot and the field or obstacles. In addition to this, we propose the method considering collisions of robots in this chapter.

3.1 Repulsive Potential When the collision-free trajectory in the environment where multiple robots exist is generated by the method described above, the computational cost increases. Because movements of robots affect each other, the construction of the configuration space, the computation the potential field and the generation of the path are calculated recursively. To reduce the computational cost, we consider the other robots as a point. The result of the Minkowski addition of the robot and the other robot expressed as a

310

M. Endo et al.

Fig. 6 Potential Field based on Superellipsoid

point becomes the same shape with the robot: the contact area is the same shape with the robot, located on the center of the other robot, oriented at the angle of the robot. In addition to this, the repulsive potential is computed to contact area not Laplace potential. With the repulsive potential, the potential is expanded as follows. U = Uc +Ur

(7)

where U c is the potential field computed on the configuration space and U r is the repulsive potential between the robot generating the trajectory and other robots. Equation (7) shows that the potential on the configuration and the repulsive potential are computed individually. Thus, because the calculation for generating the configuration space and computing the potential filed on the configuration space is once, the computational cost is reduced. From (7), we generate the potential field. By using the steepest descent method derived as (6), the collision-free trajectory of the robot is generated.

3.2 Potential Shaped Superellipsoid on Each Robot Because the robot is rectangle-shaped as shown in Fig.1, a circular potential is not suitable for the potential applied to each robot. The rectangle potential is same shape with the robot but is also not suitable, because it brings the deadlock during the operation easily. Thus, in this paper, the superellipsoid is used for the repulsive potential on each robot. In [4], the superellipsoid is used for the potential applied to obstacles such as static objects. In this paper, the superellipsoid is used for the potential applied to the other robot such as dynamic object. The superellipsoid potential shaped rectangle at near the robot and circle at far from the robot. The equation of superellipsoid is expressed as follow:  x 2n r

a

2  yr 2n b + =1 a b

(8)

Trajectory Generation for Multiple Robots

311

where, xr , yr are the position as respect to the coordinate system which the origin is putted on the center of the robot, x-axis is paralleled to a side of the robot. Constants a and b is defined as follows using x-axial length 2w and y-axial length 2h of the robot. 1

a = 2 2n w 1 2n

b=2 h

(9) (10)

When n changes from 1 to ∞, the shape of the superellipsoid changes from the rectangle to the circle. For defining the potential shaped superellipsoid, we use the distance from the side of the robot K as follows. K(x, y, θ ) =

 x 2n r

a

2  2n1 yr 2n b −1 + a b

(11)

Using this distance K, we define n as follows. n=

1 1 − eα K

(12)

where, α is a parameter to change the gradient of the potential. K and n are calculated from (11) and (12) on the numerical computation. The potential shaped superellipsoid applied to robots is derived as a following equation and becomes as shown in Fig.6. U r (K) = A

e− α K K

(13)

where A is a parameter to decide the shape of the potential. Because the distance K is changed in accordance with (x, y, θ ), the potential U r is changed in accordance with (x, y, θ ). By applying the repulsive potential shaped superellipsoid on each robot, the collision of two robots is avoided. However, occasionally the deadlock is occurred using this method, because the Laplace potential does not have a local minum value other than the destination, but by adding the repulsive potential to the Laplace potential, it is posible that the local minimum value occurs. When the deadlock is occurred, trajectories of two robots are generated again with the time-delay in beginning of the calculation for avoiding the deadlock.

4 Experiments Experiments were done using experimental systems shown in Fig.1 proposed in [1]. The size of robots is 2300 × 1100 × 700[mm], and robots have the omni-directional mechanism.

312

M. Endo et al.

Fig. 7 Experiment for Collision-free Trajectory Generation

Fig. 8 Desired Trajectories of two robots

0.00 [sec]

5.75 [sec]

11.50 [sec]

17.25 [sec]

23.00 [sec]

28.75 [sec]

34.50 [sec]

40.25 [sec]

Fig. 9 Sequential Photographs of the Experiment for the Collision-free Trajectory Generation

4.1 Collision-Free Trajectory Generation Two robots switch their position with each other. The beginning position and orientation of the robots are x = 1.35[m], y = 0.81[m], θ = 180[deg] and x = 4.05[m], y = 3.78[m], θ = 0[deg] as shown in Fig.7, respectively. Collision-free trajectories of two robots are generated using the algorithm described above. Fig.8 shows the generated trajectory and Fig.9 shows sequential photographs during the experiment. A grid of the configuration space sizes 0.054 × 0.052 [m] × 1[deg]. As shown in these results, two robots switch their position completely with each other avoiding the collisition with each other.

4.2 Trajectory Generation for the Car Transportation System Using the described algorithm, approaching trajectories to the car (1100 × 600 × 1090 [mm]) are generated. As proposed in [1], two robots need to approach from

Trajectory Generation for Multiple Robots

313

Fig. 10 Trajectories Approaching the Car

0.0 [sec]

9.4 [sec]

18.8 [sec]

28.3 [sec]

37.6 [sec]

47.0 [sec]

56.4 [sec]

65.8 [sec]

Fig. 11 Sequential Photographs of the Experiment for Approachin the car

the right/left side of the car for lifting up that. The position and the orientation of the car, which treated as an obstacle, are calculated based on information of laser range finders placed in environment. Based on positions of wheels measured by laser range finders, the position and the orientation of the car are calculated. Destinations of robots are the right/left side of the car at 0.6[m] interval respectively, and calculated based on the position and the orientation of the car. Let starting positions of the robots are given. Fig.10 shows the trajectory of each robot and the position and the orientation of the car and Fig.11 shows sequential photographs during the approach. A grid of the configuration space sizes 0.1 × 0.1 [m] × 1[deg]. As shown in results, two robots approached the right/left side of the car respectively.

5 Conclusion In this paper, we described the method to generate the collision-free trajectory of robots in environment where multiple robots exist. The trajectory is generated based on the potential method in the configuration space. To avoid the collision of robots,

314

M. Endo et al.

the repulsive potential shaped superellipsoid is used. Described algorithms were integrated to iCART and experimental results illustrated the validity of the proposed method.

References 1. Endo, M., Hirose, K., Hirata, Y., Kosuge, K., Kanbayashi, T., Oomoto, M., Akune, K., Arai, H., Shinozuka, H., Suzuki, K.: A Car Transportation System by Multiple Mobile Robots -iCART. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (2008) (to appear) 2. Sato, K.: Deadlock-free motion planning using the Laplace potential field. Advanced Robotics 7(5), 449–461 (1993) 3. Lee, N., Kim, M.: Polynomial/Rational Approximation of Minkowski Sum Boundary Curves. Graphical Models and Image Processing 60(2), 136–165 (1998) 4. Volpe, R., Khosla, P.: Manipulator Control with Superquadric Artificial Potential Functions: Theory and Experiments. IEEE Transactions on Systems, Man, and Cybernetics 20(6) (November/December 1990) 5. Latombe, J.-C.: Robot Motion Planning. Kluwer Academic Publishers, Dordrecht (1991)

Distributed Control and Coordination of Cooperative Mobile Manipulator Systems Enrico Simetti, Alessio Turetta, and Giuseppe Casalino

Abstract. The paper deals with the problem of suitably coordinating the movement of two mobile manipulators in order to accomplish a given task in a cooperative way, avoiding the arms’ singular postures and joint limits. A dynamic programming approach is used for accomplishing such avoidance specifications.

1 Introduction In the robotic literature, the field of mobile manipulators has received a certain amount of attention since the beginning of the nineties, with the obvious objective of suitably exploiting the extra degree of freedom offered by the moving base, for accomplishing tasks that otherwise could not be executed completely. At the best of authors’ knowledge, one of the first paper where the problem of simultaneous coordination of locomotion and manipulation was proposed is [15]; in such a work the concept of manipulability measure (MM; see [18], [19], [17], [12]) was explicitly used for assigning a “preferred posture” to the manipulator. Then the manipulator was joint controlled for maintaining such a posture, while an additional joint velocity command (translating a desired Cartesian velocity for the end-effector) was superimposed for driving the arm to a desired location. So doing, a decrement of the arm MM was eventually provoked, and it was then compensated via the addition of a proper linear velocity provided by the platform itself. As it can be noted, such an approach was implicitly a decentralized one, in the sense that two independently designed controllers (i.e. one for the arm and one for the vehicle) were together producing coordinated actions, via an implicit underlying data exchange between them. Thus, it is just in this sense that such pioneering work [15] still turns out to be different from most of the main successive ones (see for instance [13], [1], [2], [16], [3], [4]). In those works, on the one side the simultaneous Enrico Simetti, Alessio Turetta, and Giuseppe Casalino DIST - University of Genoa, Via Opera Pia 13, Genoa, Italy e-mail: simetti,turetta,[email protected]

316

E. Simetti et al.

manipulation and locomotion problem was eventually systematically solved; on the other side this was done on the basis of centralized control approaches, where the vehicle plus the supported manipulator were seen together as a unique enlarged robotic structure. The more complex case of multi-arm mobile manipulators (also considered in some of the previously cited works) rapidly brought into evidence the need of producing scalable designs enabling to deal with kinematic structures of increasing complexity. Other interesting successive works [14] approached the multirobot coordination problem by introducing role assignment mechanisms among the team members. The proposed solutions are effective in terms of coordination performances, but the presence of roles (even if online dynamically exchanged [9]) makes such strategies not too different than a centralized one. Indeed the leader online decides its trajectory on the basis of the current parameters of all the other members, and then the others just receive instructions from the leader on their desired motions and move accordingly. Motivated by the above consideration, while aiming to preserve a completely decentralized approach to the problem, some of the past papers of the authors were aimed at developing a general approach to the decentralized control and coordination of mobile manipulators ([5], [6], [7], [8]). In the present work, the previous theory is first re-interpreted by applying Dynamic Programming (DP) methods; i.e. by making run (at every sampling interval) a dual step DP-based procedure along the chain constituted by the arm and the vehicle. Then, following the above re-interpretation, the DP approach to control and coordination is subsequently proposed also to deal with multi-mobile manipulators, cooperatively accomplishing the transportation and manipulation of a shared load. To this aims, the paper is therefore organized as follows: in Sect. 2 the mentioned theory for a single mobile manipulator is revised in terms of its equivalent DPbased solution; in Sect. 3 the DP approach is extended to the more complex case of cooperative mobile manipulators; some simulation results are presented in Sect. 4, confirming the validity of the proposed approach. Finally Sect. 5 draws some conclusions together with some indications for further future research directions.

2 Control of a Mobile Manipulator with Singularity and Joint Limits Avoidance Let us first consider the case of a single mobile manipulator constituted by a redundant 7 d.o.f manipulator mounted on a 2D non-honolomic car-like vehicle. For such typical robotic structure a fundamental control problem is defined in terms of making the arm end-effector tracking any given Cartesian velocity reference x˙ ∈ R6 . The assumed arm redundancy allows to state that, whenever the arm is far from kinematic singularities and joint limits, any velocity reference can be fully accomplished by a motion of the arm only. Moving from such a consideration, it is then convenient to introduce two additional tasks in charge of maintaining the arm far from reduced-mobility postures. Before presenting the proposed solution, the three tasks will be separately introduced and formally defined in the following sub-section.

Distributed Control and Coordination of Cooperative MMS

317

2.1 Task Definitions 2.1.1

The End-Effector Motion Task

According to intuition, one of the main concerns of the control should be that of minimizing the following error  2 min x˙ − x˙ (1) q˙

where x˙ is the actual Cartesian velocity of the arm end-effector. In our case we have two different contributions to the end effector Cartesian velocity; namely x˙ = x˙1 + x˙2 = J1 q˙1 + J2 q˙2

(2)

where J1 is the vehicle’s Jacobian matrix of end-effector frame < e > w.r.t. world frame < 0 > and J2 is similarly defined for the arm. 2.1.2

The Singularity Avoidance Task

The first step in order to possibly avoid singular postures is to detect them in the joint space. Yoshikawa [17] has proposed a continuous quantity that represents the distance from singular postures, defined as follows    (3) µ = det J2 J2T where µ results into a non negative scalar that becomes null only when the Jacobian matrix is not full-rank. Then by considering its time derivative

µ˙ =

∂µ ∂µ ∂q =  pq˙ ∂t ∂q ∂t

(4)

where row vector p can be efficiently real time evaluated via the procedure developed in [11], we can immediately see that any assignment to the joint velocity vector that would be capable of satisfying the following condition

µ˙ = λµ µ  µ˙ ,

λµ > 0

(5)

would lead to an increasing behavior of µ and thus inducing a motion of the arm away from the singularity. This leads to the creation of a singularity avoidance task aimed to minimize the following cost  2 min µ˙ − pq˙ (6) q˙

2.1.3

The Joint Limits Avoidance Task

Most of the robotic joints are subject to excursion constraints that make the classical pseudo inversion solution of the velocity task unsuitable. Thus, in order to overcome

318

E. Simetti et al.

this problem let us introduce the following cost functional (similar to what proposed in [10])  n  n (qi max − qi min )2 1 −1 (7) h(q) = ∑ hi (qi ) = ∑ i=1 4 (qi max − qi )(qi − qi min ) i=1 where n is the number of arm’s joints. According to intuition, cost h(q) goes to infinity iff at least one of the joints approach its limits. Similarly to what we done before it is easy to see that if the joint velocity vector would be capable of satisfying the following condition ˙ h˙ = −λh h  h,

λh > 0

(8)

we would have a decreasing behavior of the cost h and thus inducing a motion of the joints away from their limits. Thus the joint limit avoidance task should be oriented to minimize the following cost function:  2   min h˙ − d q˙

(9)

q˙

where d is the gradient of h. 2.1.4

Combined Avoidance Tasks

It is possible to combine (6) and (9) into the following more compact expression: ˙ − Aq˙2)2 minq˙2 R(w

(10)

where R  diag(β1 , β2 ) ;

 w˙  µ˙

T h˙

;

 A  pT

dT

T

(11)

being β1 and β2 weighting factors online selected according to contingent needs.

2.2 A Distributed Backward-Forward Coordination Strategy As earlier anticipated, the basic rationale behind the proposed control solution is to maintain the manipulator as far as possible from all the postures characterized by a reduced mobility. To this aim, the idea is to exploit the arm extra d.o.f. induced by its redundancy and the presence of the moving base for solving an optimization problem aimed to accomplish both the singularity and the joint limit avoidance tasks, while not perturbing (whenever possible) the execution of the original velocity tracking task.

Distributed Control and Coordination of Cooperative MMS

2.2.1

319

Arm Control Law

By introducing the dynamic system ⎧ ⎨ ε˙2 = ε˙1 − J2 q˙2 ε˙1 = ε˙0 − J1 q˙1 ⎩˙ ε0 = x˙

(12)

we can reformulate the main task (1) as the following minimization problem  2 min x˙ − x˙ = min ε˙2 2

(13)

q˙1 ,q˙2

q˙

which, by applying well known DP techniques, can be solved as follows   2 2 min ε˙2  = min min ε˙2   min ε˙2 | ε˙1 2 = min ε˙1 − J2q˙2 2 q˙1 ,q˙2

q˙2

q˙1

q˙2

q˙2

(14)

This corresponds to first solving the minimization problem in the unknown q˙2 (obtaining a parameterized solution by vector ε˙1 ) and then finding out the most suitable value for vector ε˙1 . The first step leads to 

q˙o2 = J2# ε˙1 + I − J2# J2 y, y ∈ Rn (15)

where y is an arbitrary vector, which can be used for trying to accomplish the secondary tasks. To this aim, by substituing (15) into (10) and by minimizing w.r.t. vector y we obtain the following expression for y maximally contributing to the secondary tasks execution   # y = RA I − J2# J2 R w˙ − AJ2#ε˙1

(16)

 #  q˙o2 = J2# ε˙1 + RA I − J2# J2 R w˙ − K ε˙1

(17)

In the light of (16), (15) finally becomes, after some maths

where we have let K  AJ2# for notation simplicity. 2.2.2

Vehicle Control Law

By substituting the arm control law (17) into the expression of the secondary tasks cost (10) we obtain the following result:  2  2 minq˙2 R(w˙ − Aq˙2) = U2 w˙ − K ε˙1  (18) having let

  #  U2  R I − A RA I − J2#J2 R

(19)

Now by using (12) and minimizing w.r.t. q˙1 we get the following vehicle control law:

320

E. Simetti et al.

 q˙o1 = − (U2 KJ1 )# U2 w˙ − K x˙

(20)

Note that rows of matrix K can be interpreted as the pair of vectors indicating the Cartesian directions of maximum increment (decrement) of the associated secondary task gradient while U2 takes into account which part of the secondary tasks can be accomplished by the arm on its own. Both these matrices are real-time computed by the arm controller and sent to the vehicle controller at each sampling interval. After having computed its own control law (20), the vehicle controller calculates the vehicle contribution to the end-effector instantaneous velocity and sends the resulting vector ε˙1  x˙ − J1 q˙1 back to the arm. 2.2.3

Remark

Note that the previous procedure does not however always guarantee the overall singularity and joint limits avoidance because the vehicle, which is characterized by a limited planar mobility space, might not be always in the condition of properly compensating for the reduction of the arm mobility. When this is the case, in order to avoid that the arm attains a singular posture or reaches a joint limit, we have to exchange the tasks priority if one of the two secondary cost terms approach its threshold value. This can be done by introducing at the arm level a Cartesian correction signal z˙ which, added to the original velocity reference ε˙1 , guarantees the accomplishment of the secondary tasks, thus preserving the arm mobility. More details on such a correction term can be found in [11] and [6]. Here is just reported the resulting form of z˙ signal:

 ˙ − K ε˙1 z˙  (Λ K)# Γ Λ w (21) where matrices Γ and Λ are   γ (µ ) 0 Γ= 1 0 γ2 (h)

;

Λ=



λ1 ( µ ) 0 0 λ2 (h)



(22)

where γi and λi are bell-shaped scalar functions. Λ defines the inversion threshold, Γ is used to get a smooth transition avoiding chattering. Equation (17) is now finally modified accordingly  #  q˙o2 = J2# (ε˙1 + z˙) + RA I − J2# J2 R w˙ − K (ε˙1 + z˙)

(23)

It is easy to see that when Γ and Λ are the identities, a full inversion is obtained and the secondary task’s error is null.

3 Coordination of Two Mobile Manipulators The case of two mobile manipulators doing a common transportation task is now considered. Subscript a and b will now indicate the two different mobile

Distributed Control and Coordination of Cooperative MMS

321

manipulators, while 1 and 2 will still refer to vehicles and arms. Let us introduce a frame < l > rigidly attached to the object. The main common task is now finding out the joint velocities solving  2 min x˙ l − Si x˙i  ,

q˙1,i ,q˙2,i

i = a, b

(24)

where x˙ l is a given Cartesian velocity reference for the load frame, Sa is the rigid body transformation matrix from end effector frame of arm “a” to frame < l >, and where Sb is similarly defined. This problem here is obviously made harder by the presence of the kinematic constraint induced by the carried load, i.e. the equation Sa x˙a = Sb x˙b must hold at any time instant To approach the problem first note how the following end-effector velocity assignment x˙ a = Sa−1 x˙ l

;

x˙ b = Sb−1 x˙ l

(25)

would satisfy the kinematic constraint whenever both the mobile manipulators were in the condition of accomplishing their assigned main task. If this is the case, all the considerations of previous section hold here leading to the same control laws seen before; i.e. (23) for the arms and (20) for the vehicles. The only difference is that now, in case at least one of the vehicle is not in the condition of preserving the manipulability of its corresponding arm, the resulting correction term z˙ cannot be computed separately by the arm controllers. Indeed, it is necessary to compute in a central way two vectors z˙a , z˙b satisfying constraint Sa z˙a = Sb z˙b = z˙ Solution to (26) is 

(26)

  −1  S z˙a ˆz = a−1 z˙  S˙ z˙b Sb

(27)

After some algebra we obtain the new priority inversion signal  

# z˙ = Λˆ Kˆ Sˆ Γˆ Λˆ w˙ˆ − Kˆ ε˙ˆ1

(28)

which is the extended form of (21) to the case of two cooperating manipulators, ˙ˆ ε˙ˆ the vertical collection of their where Λˆ , Γˆ , Kˆ are the diagonal collection and w, 1 z˙

ARM a

ARM b ˙ b , ε˙1,b Sb ,Λb , Kb ,Γb , w

U2,a , Ka , Sa

ε˙1,a

VEHICLE a

Fig. 1 Data exchange between Master A and Slave B

x˙ a

U2,b , Kb , Sb

ε˙1,b

VEHICLE b x˙ b

322

E. Simetti et al.

Fig. 2 Time behavior of Γa e Γb

Fig. 3 A simulation of a long range task executed by two mobile manipulators

corresponding term in the single manipulator case. Note that there is no need to change the vehicle control law. Figure 1 shows the data exchange that takes place between the manipulators and also between the arms and vehicles.

4 Simulative Results In this section we will show some of the simulative results we obtained in a scenario of long range manipulation with two mobile robots. In this simulation the position reference for the object frame is located outside the reachable space of the two manipulators, thus allowing the vehicles coming into play. Moreover, in some situations the reference results in positions that are not reachable at all, hence allowing to test the efficiency of the suitably introduced priority inversion technique. As can be seen in Fig. 2 the matrix Γ (consequently the signal z˙) is not null only when the reference is located an unreachable position. Figure 3 shows the movements done by the two manipulators in a long range scenario simulation, trying to give an idea of the action done by the vehicles in order to support the arms during the task execution. In Fig. 4 it is possible to see the time behavior of the two MMs, seeing how they always are above the assigned lower bound (0.07), while going under the priority inversion bound (0.14) only in the two cases where the reference is unreachable, while ha and hb are never above the assigned priority inversion bound (0.05); thus there is never need for priority inversion due to joint limit avoidance.

Distributed Control and Coordination of Cooperative MMS

(a)

323

(b)

Fig. 4 Long range simulation (a) µa (normal) and µb (dashed) (b) ha (normal) and hb (dashed)

5 Conclusion and Future Work The paper proposes a distributed control law for a mobile manipulator, enabling the execution of a common task, whilst keeping the arm far from singularities and joint limits. The technique has been extended to the case of two cooperating mobile manipulators performing a transportation task. Coordination is obtained by making one of the robots computing a priority inversion signal, while everything else is distributed. Current research work is focusing on the “tip-over” stability control and on obstacle avoidance. Acknowledgements. This research has been partially supported by MIUR, Italy (Progetto PRIN 2007 - Tematiche di controllo in celle robotizzate iperflessibili).

References 1. Antonelli, G., Chiaverini, S.: A fuzzy approach to redundancy resolution for underwater vehicle-manipulator systems. Control Engineering Practice 11, 445–452 (2003) 2. Antonelli, G., Chiaverini, S.: Fuzzy redundancy resolution and motion coordination for underwater vehicle-manipulator systems. IEEE Trans. on Fuzzy Systems 11(1), 109–120 (2003) 3. Bayle, B., Fourquet, J.Y., Renaud, M.: Manipulability analysis for mobile manipulators. In: Proc. IEEE Int. Conf. on Robotics and Automation, vol. 2, pp. 1251–1256 (2001) 4. Bayle, B., Fourquet, J.Y., Renaud, M.: Using manipulability with non-holonomic mobile manipulators. In: Int. Conf. on Field and Service Robotics, pp. 343–348 (2001) 5. Casalino, G., Indiveri, G., Turetta, A., Aicardi, M.: Vehicle manoeuvring and multiarm motion coordination within grasping operations. In: Mediterranean Conference on Control and Automation (July 2002) 6. Casalino, G., Turetta, A.: Coordination and control of multiarm, non-holonomic mobile manipulators. In: Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS 2003), vol. 3, pp. 2203–2210 (2003) 7. Casalino, G., Turetta, A.: Coordination and control of multiarm nonholonomic mobile manipulators. In: Siciliano, B., De Luca, A., Melchiorri, C., Casalino, G. (eds.) Advances in Control of Articulated and Mobile Robots. STAR, vol. 10, pp. 171–190 (2004)

324

E. Simetti et al.

8. Casalino, G., Turetta, A., Sorbara, A.: Computationally distributed control and coordination architectures for underwater reconfigurable free-flying multi-manipulator. In: Workshop on Underwater Robotics, Genova, Italy (November 2005) 9. Chaimowicz, L., Kumar, V., Campos, M.F.M.: A mechanism for dynamic coordination of multiple robots. Autonomous Robots 17(1), 7–21 (2004) 10. Chan, T.F., Dubey, R.V.: A weighted least-norm solution based scheme for avoiding joint limits for redundant joint manipulators. IEEE Trans. on Robotics and Automation 11(2), 286–292 (1995) 11. Marani, G., Kim, J., Yuh, J., Chung, W.K.: A real-time approach for singularity avoidance in resolved motion rate control of robotic manipulators. In: Proc. IEEE Int. Conf. on Robotics and Automation, vol. 2, pp. 1973–1978 (2002) 12. Nakamura, Y.: Advanced Robotics: Redundancy and Optimization. Addison-Wesley, Reading (1991) 13. Nassal, U.M.: Motion coordination and reactive control of autonomous multimanipulator systems. Journal of Robotic Systems 13, 737–754 (1996) 14. Sugar, T.G., Kumar, V.: Control of cooperating mobile manipulators. IEEE Trans. on Robotics and Automation 18(1), 94–103 (2002) 15. Yamamoto, Y., Yun, X.: Coordinating locomotion and manipulation of a mobile manipulator. IEEE Trans. on Automatic Control 39, 1326–1332 (1994) 16. Yamamoto, Y., Yun, X.: Unified analysis on mobility and manipulability of mobile manipulators. In: Proc. IEEE Int. Conf. on Robotics and Automation, vol. 2, pp. 1200–1206 (1999) 17. Yoshikawa, T.: Analysis and control of robot manipulators with redundancy. In: Brady, M., Paul, R. (eds.) Robotic Research: The First International Symposium, pp. 735–747. MIT Press, Cambridge (1984) 18. Yoshikawa, T.: Manipulability of robotic mechanisms. Int. J. of Rob. Research 4(1), 3–9 (1985) 19. Yoshikawa, T.: Foundations of Robotics: Analysis and Control. MIT Press, Cambridge (1990)

Rearrangement Task by Multiple Robots Using a Territorial Approach Norisuke Fujii, Reiko Inoue, and Jun Ota

1

Abstract. In this paper, we address a rearrangement task involving multiple robots. We focus on small working environments composed of narrow corridors. In such an environment, interference between robots resulting from a competition for physical space can affect the overall performance in a rearrangement task. Because of the enormous amount of calculation time required to develop a motion plan for each robot considering the effect of interference, it is difficult to apply the former approach directly. In this paper, we propose a practical method using a territorial approach where the working area of each robot is defined so that there is no overlap. Because robots can develop a motion plan with less consideration of interference between other robots, the entire calculation time can be reduced. In this method, we obtain the order of sub-task executions as a solution to a project scheduling problem. However, some constraints that have not been dealt with in research involving scheduling may arise. We analyzed these new constraints and proposed a metaheuristic strategy to determine the order of sub-tasks. The proposed method is tested in a simulated environment with up to four mobile robots, 12 movable objects, and a complicated arrangement of objects.

1 Introduction Rearrangement tasks involving several objects are fundamental for mobile robots. Robots transport movable objects from an initial position to a goal position. An example of a rearrangement task is shown in Fig. 1. These kinds of tasks have various applications, such as production systems, hazardous waste cleanup, and household maintenance. In many of such applications, the working environments are small and involve narrow corridors. In such an environment, interference between robots resulting from a competition for physical space can affect the overall performance of the task. An enormous amount of calculation time is required to develop a motion plan for each robot while considering the effects of interference. Norisuke Fujii, Reiko Inoue, and Jun Ota Department of Precision Engineering, The University of Tokyo e-mail: {norisuke,reiko,ota}@robot.t.u-tokyo.ac.jp

326

N. Fujii et al.

In related research, the focus has been on the involvement of multiple robots in rearrangement tasks. One “task,” for example, the transportation of an object, is allocated to each robot, and the robots independently develop motion plans for their allocated tasks (Cambon and Grabot 2003). However, in such studies, the working environment is assumed to be sufficiently open or to contain refuge areas, which results in less interference. There are some other research fields that discuss task realization involving multiple robots and narrow corridors. In some studies, an entire working environment is divided into several areas with no overlap, and robots operate in each area (Trevai et al. 2008), (Gerkey and Mataric 2004), (Fontan and Mataric 1998). A transportation task involving objects in a narrow working environment is also the focus of some studies (Fukuda et al. 1996). In these studies, the order of task execution is easily obtained or pre-defined. It is because there are no task constraints or the initial or goal configurations of all objects are the same. The purpose of this paper is to propose a practical method for a re-arrangement task that obtains a suboptimal solution in a feasible calculation time. The major challenge is to focus on a complicated working environment that is small and contains narrow corridors. One scheme to achieve feasibility is to utilize a territorial approach. In other words, the working area of each robot is defined so that there is no overlap (referred to here as territory), and each object is transferred from one robot to another at configurations bordering these regions. This transference is called delivery, and configurations in which an object is transferred are called delivery positions. By using the above strategy, each robot can execute each delivery with less consideration of interference from other robots. Therefore, a rearrangement task can be treated as one variation of a project scheduling problem in which one activity is a delivery (called a task).

Fig. 1 Example of a rearrangement task. 3 robots and 10 objects are involved

Another scheme is to focus on rearrangement tasks that contain constraints on the order of transportation. For example, in Fig. 1, object 7 must be moved away from the initial configuration before object 6 is grasped. This is because the transportation path for object 6 is blocked by object 7. In this study, these concerns are referred to as task constraints. Considering these task constraints can filter out many invalid transportation orders and reduces the required calculation time. In

Rearrangement Task by Multiple Robots Using a Territorial Approach

327

contrast to scheduling problems, in rearrangement problems, task constraints are not pre-defined by designer. Therefore, calculation procedures for task constraints should be programmed along with the method for generating transportation orders following task constraints. The method is proposed on the basis of a metaheuristic strategy. Because it is easy to set up and handle parameters, we adopted a simulated annealing (SA) as metaheuristics.

2 Problem Description and Assumptions Alami et al. formulated a rearrangement problem for one robot (Alami et al. 1995). Their definitions and formulations have been expanded as follows to cope with a multi-robot case: Given a description of a working environment, the initial configurations, and the goal configurations of objects, find the all manipulation paths of robots. A manipulation path is a path in a configuration space that consists of a finite sequence of the path along which an object is transported and the path on which the object is placed. In this paper, the following assumptions are made:

• With any one object, two grasping configurations are set. One is the front of the • • • •

object, and the other is the back of it. All robots are equipped two wheels and one gripper for grasping an object. All robots are equipped to locate objects. Communication among robots is conducted whenever necessary. Each robot can grasp only one object at a time.

3 Rearrangement Method 3.1 Overview The proposed method contains three stages The processes in each stage are as follows: 1. The establishment phase of each territory and delivery positions. 2. The planning phase of the transportation orders following task constraints. 3. The manipulation path development phase following the final order.

3.2 Establishment of Territory and Delivery Positions In this phase, the location of the delivery positions is determined. First, the entire working environment is divided into several pieces using the construction method of a discrete Voronoi diagram (Murashima et al. 1998). Next, the territories of each robot are constructed from these pieces. Finally, delivery positions are set up on the borders of the territories.

328

N. Fujii et al.

At the first of this phase, the entire working environment is divided into several pieces using a construction method for a discrete Voronoi diagram. For example, the working environment in Fig. 1 is divided as shown in Fig. 2 (a). Each piece contains one mother points which indicates the initial or goal configurations of objects. For all configurations contained in the same pieces, mother point of the piece is closest among all initial or goal configurations of object. Distances from mother points to each configuration indicate the length of the moving path for robots.

(a) Working environment division. Colors indicate the same pieces, and “iX” and “gX” indicate the initial and goal configurations of object X.

(b) Territories and the initial configu-(c) Delivery positions. ration of a robot. Fig. 2 Example of delivery positions

Next, the territories of each robot are constructed from pieces. An example of territories is shown in Fig. 2 (b). Each territory is composed of pieces which is neighboring to the initial configuration of the same robot. Based on the territories, delivery positions are established. One criterion for the design of these positions is that regions occupied by objects at delivery positions should not overlap. This is because such overlaps increase the interference between tasks resulting from a competition for physical space, which makes it difficult to find feasible solutions. In summary, the objective of the process in this phase is to establish nonoverlapping delivery positions. If such positions cannot be established, the process results in the creation of overlapping positions. An example of delivery positions is shown in Fig. 2 (c). Because the corridor is narrow, all delivery positions are created with overlaps. With the creation of delivery positions, it is possible to determine which robot will transport which object to what location. The starting point of transportation is the initial configuration or delivery position, and the arrival point is the goal configuration or delivery position.

Rearrangement Task by Multiple Robots Using a Territorial Approach

329

3.3 Planning of Transportation Orders At this stage, a rearrangement problem can be treated as a variation of a project scheduling problem. That is, an activity corresponds to a task; a renewable resource corresponds to each robot; and precedence relations among activities correspond to task constraints. In a rearrangement task, a nonrenewable resource is not involved. Task constraints in a rearrangement task, unlike those in a project scheduling problem, are not provided by designers, as reported in a previous subsection. In this phase, transportation orders following task constraints are planned. First, in this phase, task constraints are calculated. Next, an initial solution calculated by TSP solver is rearranged with SA. Task constraints are considered in final rearrangement process; therefore, final orders may observe task constraints. The fist step of this phase is to calculate task constraints. One solution to find all task constraints is to utilize a partial order planner. Unlike the case of the problems in AI plan-space planning, the preconditions of each task are not given, and robots must begin by searching for them in a rearrangement task. However, the number of possible states in a planning space is exponential to the number of tasks, and it is unfeasible to scan all states. Fortunately, some states that do not fulfill the preconditions can be determined by using several parameters (the states of several tasks), and some task constraints can be identified by paying attention to them. Here, the symbol Ti , j is defined as the transportation of object j by robot i. The task constraints dealt with here are classified into three groups. The first is constraints related to delivery. Naturally, pick-up of an object at a delivery position cannot be executed before the delivery of that object is completed. Therefore, we can define the first group of constraints as T A, j p TB , j , where A is the transferring robot and B is the receiving one. The second is constraints related to overlaps of delivery positions. The order of delivery for objects whose delivery positions interfere must be identical among the robots. For example, in Fig. 3, two alternatives of task constraints can be calculated. In other words, (T0,0 p T0,1 ) ∧ (T1,0 p T1,1 ) or (T0,0 f T0,1 ) ∧ (T1,0 f T1,1 ) should be observed. If this kind of constraint is ignored, two robots could collide in the vicinity of the delivery position, or both robots could be kept indefinitely waiting for delivery. Finally, the constraints prescribed by the arrangement of objects should be considered. These kinds of constraints are calculated as follows:

• Robot i tries to generate a transfer path to transport object A assuming that there are no other objects with the exception of object B at the initial configuration. If there is no path, Ti , A f Ti , B . • In the same way, if there is no path for robot i to transport object A while only considering object B at the goal configuration, Ti , A p Ti, B .

330

N. Fujii et al.

Fig. 3 A case in which two delivery positions overlap

After the completion of calculating task constraints, each robot calculates the initial order of transportation. In this process, robots do not consider the task constraints and treat the rearrangement problem as TSP, where the node indicates one state of tasks and the edge indicates the transition from one node to another. In our method, the cost of the edges is estimated by the length of the manipulation path without consideration of the other objects. Next, the initial orders are rearranged with SA considering the task constraints. In our method, the search space of SA contains an invalid solution, and a large penalty is added to the cost function of an invalid solution. This is because, in a rearrangement task, the design of the search space that contains only an executable solution is difficult. With our method, one state in the search space is defined as pairs of transportation orders for each robot. For example, a collection of permutations {|1, 0|, |0, 1|} means that robot 0 executes transportation in the order of objects 1, 0 and robot 1 transports in the order of objects 0, 1. A neighborhood can be defined as one operation “exchanging the transportation order of the i-th and j-th object for robot k.” The cost for one state is given by the time of the end of the last transportation (makespan). This makespan can be calculated by deciding the start times of each transportation so that it may follow rearranged orders and observe all task constraints. The time spent for one task is estimated by {(the length of transportation path calculated in the former subsection / the speed of the robots)}. In addition, if the generated order violates the task constraints, a large value is added to the cost function. The penalty value is set so that it is much larger than the general cost function.

3.4 Manipulation Path Development In this phase, manipulation paths for transportation tasks are developed by following the final order. Robots develop a path for picking up an object and a path for placing the object.

Rearrangement Task by Multiple Robots Using a Territorial Approach

331

4 Simulation Simulations are conducted for 8 problems involving up to 4 robots and 12 objects. The CPU is Core Duo, 1.06GHz. In simulations, robots are assumed to move at 0.1m/sec. The time limit of the modification phase of the transportation order (mentioned in Subsection 3.3.3) is 60 seconds. As cooling schedule of SA, we adopt popular one. That is, temperature parameter t(i) of i-th cycle is derived from following equation; t(i) = t(i-1)*0.9. The number of iteration times N(t) in a certain temperature t is derived from following equation; N(t) = N(t(i-1))*1.05. We tested the proposed methods and former method. For each method, we carried out 5 simulations for one problem, and the total number of simulation times for one method was 40; then, we evaluated the averages. The former method works as follows: each robot selects the closest eligible object, tries to transport the selected object, and re-selects another eligible object if the manipulation path for the current object is blocked by other robots. Each robot determines which objects are eligible depending on whether there are preceding objects (Fujii and Ota, to be published).

(a) Initial.

(b) 15 seconds.

(c) 30 seconds.

(d) 45 seconds.

(e) 60 seconds.

(f) Goal.

Fig. 4 Simulation results in which 3 robots and 10 objects are involved

An example of simulation results is shown in Fig. 4. These figures show that robots 0 and 1 do not go out of their rooms because borders are constructed. Robot 0 delivers objects 0 at first to the robot 2 because objects 0 block other manipulation paths (Fig. 4(b)). It is evident that task constraints are correctly calculated and most prior objects are properly chosen. Delivery of object 4 and that of objects 3 require interfered delivery positions, but robots do not deliver them at the same time. This shows that the competition of physical space is correctly coordinated. Table 1 shows the percentages of feasible solutions. While the former method fails to find feasible solution for the half of tested method, our proposed method reaches feasible solution for all tested environments. Environments that the former method is not applicable, such as shown in Fig. 4, include narrow corridors that do

332

N. Fujii et al.

not permit a robot to overtake another. Therefore, the former method tends to fall into a deadlock. To evaluate these results more briefly, we define the difficulty for each working environment as follows:

D = 1 /(d w * 2 − max ( w Ri , wMj )) + 0.5 * m + 0.1 * n i, j

(4.1)

Here, D is the difficulty of environment, w Ri is the width of robot Ri , wMj is the width of object Mj , m is the number of robot, and n is the number of object.

d w is the average of minimum distance to wall from the border of voronoi region for line segments which indicate walls. d w * 2 represents width of corridors, therefore the denominator of first term indicate spatial room of corridors. We regard the reciprocal of these rooms as the special difficulty of environment. It seems that the more robots are involved, the more complicated the environment become. Therefore, we define the simple summation as difficulty of working environment. Fig. 5 shows the relation between percentages of feasible solutions and difficulty. From this result, former method able to find feasible solution as far as difficulty is less than 2.5. On the other hand, ignore-alternatives method is applicable if difficulty is more than 3.5. We also compared the solution qualities as deviations of the task execution times from the lower bounds which are shown in Table 1. We set the lower bound as follows: (the total path length for placing an object) / (the number of robots). This is because the length of the path for taking an object can change depending on the locations of the delivery positions and can decrease to zero in the case of an ideal situation. Table 2 also indicates the efficiency of the proposed method. Using former method, robots seem to waste times for collision avoidance. Finally, the average of the total calculation time is 163 [sec]. The total calculation time contains time for territory establishment, planning of the transportation order, and development of transportation paths. This result shows that our method can achieve the objective of this paper, as reported in Section I, and the practicality of the proposed method.

Fig. 5 Relation between feasibilities and difficulty of environments

Rearrangement Task by Multiple Robots Using a Territorial Approach

333

Table 1 Deviations from lower bounds [%]

Method \ environment Proposed method Former method

I II III IV V VI VII VIII average 458 337 441 354 358 369 402 402 390 508 392 359 470 432

5 Conclusion In this paper, we propose a method for the realization of a rearrangement task. Our method defines several working areas for robots that do not overlap and delivery positions. Our method solves a difficult problem, and former approaches may not be applicable. At the same time, calculations are finished in feasible computational time. On the other hand, most tasks concentrate on a robot in some cases. Therefore, the set-up procedure of the working region can be improved. Further research will be conducted to explore some of these directions in more depth. This work was partially supported by the Yazaki Memorial Foundation for Science and Technology and supported in part by Global COE Program “Global Center of Excellence for Mechanical Systems Innovation”, MEXT, Japan.

References Alami, R., Laumond, J.P., Simeon, T.: Two manipulation planning algorithms. In: Goldberg, K., Halpern, D., Latombe, J.C., Wolson, R. (eds.) Algorithmic Foundation of Robotics, pp. 109–125. AK Peters, Boston (1995) Cambon, S., Grabot, F., Alami, R.: aSyMov: Toward more realistic robot plans. In: Rapport LAAS N° 03472. LAAS, Toulouse, pp. 1–8 (October 2003) Fontan, M.S., Mataric, M.J.: Territorial multi-robot task division. IEEE Trans. on Robotics and Automat. 14, 815–822 (1998) Fukuda, T., Ishihara, H., Hiraoka, N.: Deadlock resolution in distributed autonomous robotic system with hand-to-hand motion. In: Proc. IEEE Int. Conf. Robotics and Automat., pp. 2049–2054 (1996) Gerkey, B.P., Mataric, M.J.: A formal analysis and taxonomy of task allocation in multirobot systems. Int. J. Robotics Research 23, 939–954 (2004) Murashima, S., Watanabe, T., Fuchida, T., Mori, K., Kurizono, M.: Construction of 2-dimentional digitized voronoi diagram based on non-euclidean distance. Trans. Institute of Electronics, Information and Communication Engineers J81-D-1, 691–699 (1998) Fujii, N., Ota, J.: Rearrangement task realization by multiple mobile robots with efficient calculation of task constraints. J. Advanced Robotics (to be published) Trevai, C., Fujii, N., Ota, J., Arai, T.: Multiple mobile robot exploration and patrol strategy using a self-organizing planner based on a reaction-diffusion equation on a graph. J. Robotics and Mechatronics 20, 24–37 (2008)

A Task Planner for an Autonomous Social Robot Samir Alili, Rachid Alami, and Vincent Montreuil

Abstract. This paper describes a high-level task planner called HATP (for Human Aware Task Planner). which is designed to synthesize plans for human robot teamwork while respecting social conventions and adopting acceptable collaborative behaviors. We provide an overall description of HATP and discuss the features of its plans evaluation mechanisms. We also illustrate its performance on a real scenario achieved by a robot in interaction with a human partner in a realistic setup.

1 Introduction One challenge in robotics research is to develop socially interactive and cooperative robots. Fong et al[8] define “Socially interactive robots” as entities which “operate as partners, peers or assistants, which means that they need to exhibit a certain degree of adaptability and flexibility to drive the interaction with a wide range of humans”. This definition was made explicit by Klein et al[11] with what they called “ten challenges for human robot teamwork”. We believe that several of these challenges can be handled at the task planning level. Indeed, the robot should be able (1) to signal in what tasks it can/wants participate, (2) to act in a predictable way to ensure human understanding of what it is doing, (3) to exhibit its status and its intentions, (4) to negotiate about tasks with its human partner in order to determine roles and select how to perform these tasks and (5) to deal with social conventions, as well as its human partner abilities and preferences. The work presented here consists of an approach, a model and an implemented planner called HATP (for Human Aware Task Planner) which has been specially designed for the interactive action between heterogeneous agents, in our case humans and robots. It is based on hierarchical task planning[10] which gives to the planner Samir Alili, Rachid Alami, and Vincent Montreuil LAAS-CNRS; University of Toulouse, 7 Avenue Colonel Roche, 31077 Toulouse Cedex 4, France e-mail: [email protected]

336

S. Alili et al.

some ability to answer to certain issues defined above. It also integrates “social behavior rules”, which orient the robot decision toward the synthesis of socially acceptable plans. Section §2 presents HATP domain representation: world, methods and actions, and explains the rules that we have chosen to favor robot behaviours that are more suitable to Human-Robot Interaction (HRI) context. Section §3 gives details of plan evaluation to obtain social behaviors. Section §4 describes the implemented system and illustrates its use through several examples involving a mobile manipulator acting in interaction with a person. Finally, section §5 draws the main conclusions and discusses future work.

2 HATP Planner Description HATP is a hierarchical task planner a problem is represented with a tuple P =< W0 , D, T T L >, where W0 is initial world state, D is a domain and T T L represents a set or tree of tasks. The domain is represented by a pair D =< T, R >, where T is a set of tasks and R is a set of rules. We can distinguish two types of tasks in T = Op∪M. Op correspond to basic primitives and M represent non-primitive tasks. R is a set of “social rules”. Each ri ∈ R is represented by tuple ri =< Bi , PeAg,ct , pri > i where Bi is the description of the rule, (i.e. description of patterns to recognize in Ag,ct is a penalty to be added to the plan score if the rule ri is the plan structure). Pei Fig. 1 World Data Base Representation in HATP: in this example, we define the entities Place and Ob ject. The entity Agent is predefined. A set of atributes is associated to it: (type, maxObjects, posTop, full, objects) which represent respectively a type of Agent (human, robot. . . ), its capacity, its current position, a boolean which indicates if the Agent has reached his max capacity, the list of objects that the agent is holding. We can see that ob jects and Place are of type Entity. The last lines illustrate the initialisation of the attributes of a robot called “Jido” and human called “Samy”

GHILQHHQWLW\7\SH 3ODFH GHILQHHQWLW\7\SH 2EMHFW GHILQHHQWLW\$WWULEXWHV$JHQW ^ VWDWLF DWRP VWULQJW\SH VWDWLF DWRP QXPEHUPD[2EMHFWV G\QDPLF DWRP ERROIXOO G\QDPLF DWRP 3ODFHSRV7RSR G\QDPLF VHW  2EMHFWREMHFWV ` -LGR QHZ$JHQW  6DP\  QHZ$JHQW 'RRU3ODFH QHZ3ODFH %RWWOH  QHZ2EMHFW 6DP\W\SH 6DP\PD[2EMHFWV -LGRW\SH -LGRPD[2EMHFWV -LGRSRV7RSR -LGRIXOO -LGRREMHFW 

KXPDQ  URERW  'RRU3ODFH  IDOVH %RWWOH

(QWLW\W\SHGHFODUDWLRQ

$WWULEXWHVGHFODUDWLRQ

(QWLW\GHFODUDWLRQ

$WWULEXWHVLQLWLDOL]DWLRQ

A Task Planner for an Autonomous Social Robot

337

violated in the plan solution P. The value depends on the agent who violates the rule and the context in which the rule has been violated. A priority pri > is associated to each rule. Let us give more details about the elements of HATP planning domain. First, we give the description of HATP syntax and a formal description of actions (also called operators) and methods. Then, we define the different forms of social rules available in HATP. World database: In HATP the world description is represented by a set of entities W b =< En1 , En2 , En3 , ..., Enn >, each entity is unique and defined by set attributes that are similar to predicates or state variables in classical planning. They can be of different types, each type is defined as tuple Att =< Ct1 ,Ct2 ,Ct3 >, where Ct1 ,Ct2 and Ct3 range respectively over the set {Static, Dynamic}1, {Atom, Set}2 and {Numeric, Boolean, String, Entity}. Figure 1 illustrates an example of a HATP domain description.3 Operators represent the actions i.e. tasks that can be directly executed. They are defined as a tuple Op =< Pecd, E,C, Du > where Pecd is a precondition formula, E is the effect on the world after action execution. C : (Ct, agent) → ℜ+ is the cost function of action execution; it depends on the agent and on the context, D ∈ ℜ+ is the duration of action execution (see Fig. 2.a). Methods represent the tasks that must be further decomposed into subtasks. They are defined as pairs M =< goal, D >. If the goal formula is true then the goal is already reached and the method is reduced to no-op4 . D = {D1 , . . . , Dn } is DFWLRQWDVNQDPH(Q(Q (Q Q ^SUHFRQGLWLRQ ^25 ^IRUPXOD `  ^IRUPXOD Q`` HIIHFWV ^IRUPXOD LI^FRQGLWLRQ` ^IRUPXODIRUWKHFRQGLWLRQ `  LI^FRQGLWLRQQ` ^IRUPXODIRUWKHFRQGLWLRQ Q`` FRVW^IRQFWLRQFRQWH[WGHSHQGHQW ` GXUDWLRQ^IRQFWLRQFRQWH[WGHSHQGHQW ``

Fig. 2 HATP method and action representation: we can note disjunctive set of preconditions and also conditional effects 1 2 3

PHWKRGWDVNQDPH(Q(Q(Q Q ^ HPSW\ ^25 ^IRUPXOD `  ^IRUPXOD Q` ` GHFRPSRVLWLRQ ^SUHFRQGLWLRQV  ^25  ^IRUPXOD`    ^IRUPXODQ`  ` VXEWDVNV ^,GWDVNQDPHSDUDPHWHU     ,GQWDVNQDPHSDUDPHWHU   ` `  GHFRPSRVLWLRQQ ^` `

Dynamic attributes can be modified during the planning process whereas Static attributes can not. Set attributes can store several values whereas Atom can have only one value at a time. The entity Agent is predefined by the system.

338

S. Alili et al.

a set of alternative decompositions. Each decomposition Di is a tuple Di =< Pecdi , T Li ,Ci > where Pecdi is a precondition, T Li ∈ T is a set of subtasks which decompose the method M according to the constraints Ci which are generally precedence constraints (see Fig. 2.b). Social Rules: We define a social rule as a convention which associates penalties to a behaviour of an agent in a given context. In human environments most of these rules are implicit between the members of a community and they vary from a society or a culture to another. In HATP we have chosen six rule types that seem to be general rules. A given domain definition may include or not any number of rules: Undesirable states correspond to world states which are dangerous, unpleasant or inadequate. Example: the robot lets the fridge door open, the robot puts an object on the floor which may impede the movement of other agents or, more dangerous, leaves the gas switched on. Undesirable sequences some action combinations can conduct to an uncomfortable feeling for robot partners. For example, a plan in which the robot puts down an object and its partner picks it up immediately after might be considered as an awkward sequence if a the robot can hand over directly the object. Bad decompositions this rule is used to select better choices at a high level of decomposition. The idea is that specific decompositions must remain possible but they must be used only if necessary. For example, when the robot has to leave an object for someone, it is better to place it on a “visible” place. Effort balancing The idea is to establish a balance of effort among partners. If we have a team of two robot agents, the amount of effort could be the same for all the staff. If the team is composed of a human and a robot, then the robot must provide more effort than the human. Timeouts the idea is to prevent long waiting time between two actions done by a same agent, because it is prejudicial for the plans quality. Intricate links between plan streams Crossed links represent precedence links between actions of two different agents. Such links make plans fragile and introduce a lot of dependency between agents.

3 Social Plan Evaluation In HATP, the planning process is composed of two threads; a refinement thread and an evaluation thread. The refinement thread algorithm is responsible for the plan searching it is largely inspired from SHOP2 procedure [13]. The main differences between them are that HATP can have as input a set of tasks but also partial trees. HATP can reduces time exploration for solution by making the assumption that if two tasks are ”parallel” 4

no-op is an Operator whose precondition is always true, whose effect is empty and which has a null duration and cost.

A Task Planner for an Autonomous Social Robot

339

(i.e. they do not have causal link between them), it will try to obtain a final plan where they are as independent as possible (for more details, refer to[12]). The evaluation thread is responsible of complete evaluation of plans; it is based on the Decision by Objectives theory, more precisely The Analytic Hierarchy Process (AHP)[9]. It is a generic method[15, 14], designed to facilitate decision making based on several heterogeneous criteria in complex environments. It decomposes the decision problem into a hierarchy of more easily comprehended sub-problems, each of which can be independently analyzed. HATP plan evaluation: The criteria considered in the plan evaluation are social rules described in the HATP domain and the cost of all actions involved in a given plan. The evaluation in HATP in performed in two phases. First, construction of the criterion dominance vector and computation of the best score solution. The first phase is done off-line. It consists in making pairwise comparisons between all criteria and in the synthesis of the dominance degree of each criterion. In order to do this, HATP uses priorities pri associated to each social rule. A priority value is between −8 and 8. 0 represents the reference point and is the default attached to the criteria cost. Then we can compute a comparison matrix A and also W = [w1 , w2 , . . . , wn ] (n represent number of criterion) the eigenvector associated to it, while respecting the technique described in [9], Each wi represents the weight of criteria i in plan evaluation (figure3). The second phase is performed during the planning process. The first solution produced by the planner is taken as a reference solution. Now we compare all the new generated solutions with it, and choose the solution that improves the plan. The improvement is calculated in the following way: let Pnew be a new solution plan ฀

฀

฀

฀ ฀

฀

฀

฀

฀

฀

฀

฀

Fig. 3 HATP plans evaluation: Let us assume a problem with three criteria α1 ,α2 ,α3 . We can construct a matrix A of pairwise comparison that provides eigenvector W = [w1 , w2 , w3 ] where wi corresponds to αi . The planning process produces a reference plan solution Pre f and a set P of all other solutions that have a null improvement. Compute the second local weight that gives three eigenvector Γ1 , Γ2 and Γ3 . In each γin i refers to a criterion and n to the solution. Then we can calculate a plan sore/quality Si (Pi ) = ∑3j=1 w j × γ ji

340

S. Alili et al. N w∗ Σi=0 i

re f Vali −Valinew re f Vali

and Pre f be reference plan, then the improvement of Pnew is . If the N value of the average is negative Pnew is immediately rejected and the process continues. Otherwise Pnew becomes the new reference. In the case where we have an null improvement, we make pairwise comparison between all solutions with respect to all criteria. We obtain several matrices and eigenvectors, the number of eigenvectors is equal to the number of criteria; the dimension depends on the number of solutions (see Fig. 3). Then, a simple linear combination between different weighting gives the score that represents the plan quality. To determine the best solution we use the formula Si (Pi ) = ∑Nj=1 w j × γ ji , which represents the plan evaluation. The best solution is the one that minimizes the score.

4 Experimental Results We illustrate below HATP performance through two scenarios. The first one is ”Fetch-And-Carry” and shows the influence of social rules on plan quality as well the capacity of HATP to handle contingency. The second scenario, named ”ServeAnd-Clean” illustrates the performance of HATP implemented on “Jido”, a mobile manipulator that dedicated to human robot interaction and cooperation [5]5 Both scenarios run in a home. We have a “living-room” with 3 types entities: Agent (robot, human), furniture (cupboard, table, sofa. . . ), and object (glass1, glass2, bottle). The robot has a symbolic as well as a geometric description of its environment. It is able to navigate robustly in its environment[7, 16], to detect and track persons[2, 3, 17, 18], and to recognize, localize and manipulate a (limited) set of objects. The overall robot architecture is described in [6, 5]. HATP interacts essentially with a robot supervisor, called SHARY[4, 6, 1]. “Fetch-And-Carry” scenario: In the initial state, the robot is at the door, the human siting is at the sofa and wants to have drink. To achieve this goal we choose to begin withe a partial plan, composed of three parts as : (1) the human must have a glass, (2) the human must have a bottle and (3) the human must reach the sofa. Besides, a specific decomposition is imposed for task (1): Jido must get the glass and give it to the human. The glasses are on the table and the bottle is in a closed cupboard. Figure 4.a illustrates the plan produced by HATP when social rules are inhibited. We can note that this plan is correct but it has a number of defects: the human opens the cupboard for the robot, the robot puts the bottle on the sofa and then the human picks it up. Now, if we introduce the following set of rules (avoid to have a bottle or glass put on the sofa, avoid to leave the cupboard open, avoid awkward sequences such that robot puts down an object and human picks it up just after, avoid that open/close sequences performed by different agents), we can see in Figure 4.b, that most defects have been corrected. However, this is not sufficient because we can 5

Some videos and pictures are available at (http://www.laas.fr/˜salili/media.php).

A Task Planner for an Autonomous Social Robot

฀

฀

฀

341

฀

฀

฀

฀

฀

฀ ฀

฀

฀

฀ ฀

฀

฀

฀

฀

฀ ฀

฀

฀

฀

฀

฀

฀

฀

฀ ฀

฀

฀ ฀

฀

฀

฀

฀ ฀

฀

฀

Fig. 4 Effect of social rules on the plans: the yellow circles represent human actions, the blue circles represent robot actions and the green ellipses represent joint actions

observe that the human moves from the sofa to the table and comes back only for getting the glass, and there is a risk to have him wait for the robot to achieve its own part of the plan. If we introduce rules that try to avoid intricate synchronisations between the robot and the human we observe (Figure 4.c) that all defects have been corrected. Now if we replace the human by a robot and change the equilibrium point for the “Effort balancing” rule , we can see the effect of these modifications in Figure 4.d. We note that the system tries to make a good balance between the agents with respect to their abilities. Now we will focus on HATP ability to handle contingency. In Figure 5.a the plan represents HATP answer for the same goal as above. If we consider the case where the object Glass1 is invisible or unreachable for Jido or Jido doesn’t know its current position, HATP will produce a plan where Glass1 is substituted by an other object of the same type Glass2 (Figure 5.b), if the two glasses are invisible to Jido, HATP produces the plan in Figure 5.c where the robot goes to last known position of the object and tries to look for it. The purple action ”Look for” represents a non-deterministic action. If the action succeeds (robot can see object) the supervisor SHARY will ask HATP for a new plan, else the supervisor will carry on in the same plan. Concerning the action ”re-plan” it is a fictive action for the supervisor. It is used in order to induce a new invocation of HATP after the execution of a

342

S. Alili et al.

฀

฀

฀

฀

฀

฀ ฀

฀

฀ ฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀

฀ ฀ ฀

฀

฀ ฀

Fig. 5 HATP failure and contingency handling: the yellow circles represent human actions, the blue circles represent robot actions and the green ellipses represent joint actions. The purple action are a non-deterministic actions except for action ”re-plan” which is a deferred planning request

non-deterministic action. In the case where the two glasses are unreachable, the robot is in a situation where it is unable to solve the problem himself. HATP produces a plan (Figure 5.d) where Jido asks the human for help. If the human agrees, the supervisor will ask HATP for a new plan (Figure 5.e) involving the human. If not, the supervisor abandons the task. Finally, if the position of the two glasses is unknown, then HATP produces a (Figure 5.f) where the robot will search for the objects in all possible positions. “Serve-And-Clean” scenario: The human asks Jido to clean a bottle that is on the living-room table. We can see on Figure 6 the ”clean-up” task execution. At the left part of the figure we see the interaction between HATP and SHARY while the right part exhibits the interaction and plans execution of the two agents: Jido and the human called H1. At the beginning H1 asks Jido to clean the yellow bottle. SHARY replies by updating HATP world database and sending a planning request (CleanUp(YELLOWBOTTLE)), HATP answers by producing a plan that Jido begins to achieve under SHARY supervision. When Jido tries to pick the bottle on the table, SHARY realizes that the bottle is unreachable. In this situation SHARY sends a new planning request. HATP produces a plan where Jido needs to interact with human to solve the situation. If H1 agrees, SHARY sends again the same

A Task Planner for an Autonomous Social Robot

฀

฀ ฀ ฀ ฀

฀ ฀ ฀ ฀

฀ ฀

฀

฀ ฀

฀

฀

฀

฀ ฀ ฀ ฀

฀

฀

฀

฀ ฀

฀

฀

฀

343

฀

฀

฀

฀

฀

฀ ฀

฀ ฀

฀

฀

฀ ฀

฀

฀

฀

฀ ฀

฀ ฀ ฀ ฀ ฀ ฀

฀ ฀ ฀

฀ ฀

฀ ฀

฀

฀ ฀ ฀ ฀ ฀ ฀ ฀ ฀

฀

฀ ฀

฀ ฀

฀ ฀

฀

฀

฀

฀

฀

฀

฀ ฀

฀ ฀

฀ ฀ ฀

฀

฀

Fig. 6 CLEAN-UP task execution: At the top left of the figure, we see a simple version of the first HATP plan computed to achieve SHARY requests. In the middle and at the right side of the figure, we see the execution stream corresponding to this plan execution. This first plan failed due to robot inability to pick the bottle (even when it has perceived it). Then SHARY asks for a new feasible plan. HATP finds a plan with a higher cost and two streams and where the person is requested to participate by giving the bottle to Jido. The robot can then proceed and move to throw the bottle in the trash bin

planning request to HATP, which produces a new plan where the H1 is asked to get and give the bottle to Jido which will then will be able to navigate and throw it in the trash bin.

5 Conclusion and Future Work We have described a robot task planner called HATP that has been designed for interactive and collaborative robotic applications. We have illustrated its ability to produce so-called “socially acceptable plans” for several agents while taking into account the agents abilities, preferences and desires and respecting social aspects related to the interaction of a robot with humans. We have seen its ability to handle contingency and to adopt, when necessary, proactive behaviors. We have also seen that, it is able to deal with partially specified plans, allowing the robot to comply with human desires and to re-plan after a failure.

344

S. Alili et al.

Concerning future work, several aspects will be investigated. The first one is the development of heuristics in the refinement process in order to explore the most promising parts of the solution space and to allow the use of HATP in more complex situations with a more elaborate model of its activities. Another aspect is about the improvement of temporal constraints management as well as the elaboration of a task representation that deals explicitly with uncertainty.

References 1. Clodic, A., Alami, R., Montreuil, V., et al.: A study of interaction between dialog and decision for human-robot collaborative task achievement. In: 16th IEEE RO-MAN (2007) 2. Breazeal, C., Edsinger, A., Fitzpatrick, P., et al.: Active vision for sociable robots. IEEE Transactions on Systems, Man and Cybernetics (2001) 3. Burger, B., Ferran´e, I., Lerasle, F.: Multimodal interaction abilities for a robot companion. In: Gasteratos, A., Vincze, M., Tsotsos, J.K. (eds.) ICVS 2008. LNCS, vol. 5008, pp. 549–558. Springer, Heidelberg (2008) 4. Clodic, A.: Supervision pour un robot interactif: Action et Interaction pour un robot autonome en environnement humain. PhD thesis, University of Toulouse (2007) 5. Clodic, A., Cao, H., Alili, S., Montreuil, V., et al.: Shary: a supervision system adapted to human-robot interaction. In: 11th ISER (2008) 6. Clodic, A., Ransan, M., Alami, R., Montreuil, V.: A management of mutual belief for human-robot interaction. In: IEEE SMC (2007) 7. Alami, R., Sisbot, E.A., Simeon, T., et al.: A human aware mobile robot motion planner. IEEE Transactions on Robotics (2007) 8. Fong, T.W., Nourbakhsh, I., Dautenhahn, K.: A survey of socially interactive robots. Robotics and Autonomous Systems (2003) 9. Forman, E., Selly, M.A.: Decision By Objectives. World Scientific, Singapore (2001) 10. Ghallab, M., Nau, D., Traverso, P.: Automated Planning - theory and practice. Morgan Kaufmann, San Francisco (2004) 11. Klein, G., Hoffman, J.M., Woods, D.D., Bradshaw, et al.: Ten challenges for making automation a ”team player” in joint human-agent activity. IEEE Intelligent Systems (2004) 12. Montreuil, V., Clodic, A., Alami, R.: Planning human centered robot activities. In: IEEE Int. SMC (2007) 13. Nau, D., Au, T.C., Ilghami, O., Kuter, U., et al.: Shop2: An htn planning system. Journal of Artificial Intelligence Research (2003) 14. Saaty, T.L.: The Analytic Hierarchy Process for Decisions in a Complex World. RWS Publications (1999) 15. Saaty, T.L.: Fundamentals of the analytic hierarchy process. RWS Publications (2000) 16. Sisbot, E.A., Marin-Urias, L.F., Alami, R.: Spatial reasoning for human robot interaction. In: IEEE/RSJ IROS (2007) 17. Lerasle, F., Simon, T., Germa, T., Br´ethes, L.: Data fusion and eigenface based tracking dedicated to a tour-guide robot. In: ICVS (2007) 18. Danes, P., Brethes, L., Germa, T., Lerasle, F.: Human/robot visual interaction for a tourguide robot. IEEE/RSJ IROS (2007)

Part V

Multi-robot Cooperation

A Stochastic Clustering Auction (SCA) for Centralized and Distributed Task Allocation in Multi-agent Teams Kai Zhang, Emmanuel G. Collins Jr., Dongqing Shi, Xiuwen Liu, and Oscar Chuy Jr.

Abstract. This paper considers the problem of optimal task allocation for heterogeneous teams, e.g., teams of heterogeneous robots or human-robot teams. It is well known that this problem is NP hard and hence computationally feasible approaches must develop an approximate solution. This paper proposes a solution via a Stochastic Clustering Auction (SCA) that uses a Markov chain search process along with simulated annealing. The original developments are for a centralized auction, which may be feasible at the beginning of a mission. The problem of developing a distributed auction is also considered. It can be shown that if the distributed auction is such that the auctioneer allocates tasks to optimize the regional cost, then the distributed auction will always decrease the global cost or have it remain constant, which provides the theoretical basis for distributed SCA. Both centralized SCA and distributed SCA are demonstrated via simulations. In addition, simulation results show that by appropriate choice of the parameter in SCA representing the rate of “temperature” decrease, the number of iterations (i.e., auction rounds) in SCA can be dramatically reduced while still achieving reasonable performance. It is also shown via simulation that in relatively few iterations (8 to 35), SCA can improve the performance of sequential or parallel auctions, which are relatively computationally inexpensive, by 6%-12%. Hence, it is complimentary to these existing auction approaches. Kai Zhang, Emmanuel G. Collins Jr., Dongqing Shi, and Oscar Chuy Jr. Center for Intelligent Systems, Control and Robotics (CISCOR), Department of Mechanical Engineering, FAMU-FSU College of Engineering, Florida A&M University-Florida State University, Tallahassee, FL 32310, USA e-mail: {zhangka,ecollins,shido,chuy}@eng.fsu.edu Xiuwen Liu Center for Applied Vision and Imaging Sciences, Department of Computer Science, Florida State University, Tallahassee, FL 32310, USA e-mail: [email protected]

346

K. Zhang et al.

1 Introduction Teams of heterogeneous agents, for example, teams of heterogeneous robots or human-robot teams are expected to be employed in a variety of applications [1, 2, 3]. The problem of coordinating these agents is known as the problem of task allocation. Current approaches to task allocation can be placed into five categories: 1) fully centralized approaches [4], 2) centralized auctions [5], 3) distributed auctions [1, 3, 6], 4) completely distributed approaches [7, 8], and 5) hybrid approaches of distributed auctions and emergent coordination [2, 3]. This paper contributes to methodologies for centralized and distributed auctions. Although the combinatorial auction provides an optimal centralized auctioning approach, it is well known to be NP complete [9] and hence becomes practically infeasible as the number of tasks and agents increase. This has led to the search for polynomial time algorithms that yield task allocations that approximate the optimal task allocation. Sandholm has suggested that probabilistic or stochastic algorithms may result in better approximation algorithms [9]. This paper presents such a stochastic algorithm for optimal task allocation; it is based on a recent algorithm that was successfully used to obtain nearly optimal clustering of planar shapes [10]. This research views optimal task allocation as optimal task clustering, i.e., determining the cluster of tasks to be assigned to each agent so that a global or regional cost function is optimized. The algorithm proposed here, called Stochastic Clustering Auction (SCA), uses a Markov chain search process along with simulated annealing to search the space of task clusters. The idea of task clustering is not new to the task allocation literature [3]. However, this research differs from previous clustering algorithms in that the clustering is directly based on optimizing a global or regional cost function by a stochastic approach. It will be shown via simulation that for centralized auctioning SCA can result in a nearly optimal task allocation while requiring reasonably small computational times. As previously mentioned, centralized auctions are primarily useful for the beginning of a mission. However, due to communication and computational requirements, they may be impractical during the mission. Hence, a second goal of this paper is to use the SCA for distributed auctioning. It can be shown that if the distributed auction is based on what has been called opportunistic centralization [3], such that the auctioneer allocates task to optimize the regional cost, then the distributed auction will always decrease the global cost. This motivates the development of a distributed auction that uses SCA to optimize the regional costs. The paper is organized as follows. Section 2 formulates the basic optimization problem for task allocation, provides a description of the Stochastic Clustering Auction algorithm, and discusses how the algorithm may be used for both centralized and distributed auctions. Section 3 presents simulation results for both centralized and distributed auctions. Finally, Section 4 presents conclusions.

A SCA for Centralized and Distributed Task Allocation in Multi-agent Teams

347

2 Stochastic Clustering Auction for Centralized and Distributed Task Allocation This section first presents the basic problem statement. It then describes the Stochastic Clustering Auction (SCA). After providing the fundamental theory of the regional cost, it is proposed that SCA be used to optimize the regional cost in a distributed auction.

2.1 Problem Statement Let H denote a set of k heterogeneous agents, and T denote a set of n tasks, i.e. H = {h1 , h2 , . . . , hk } and T = {t1 ,t2 , . . . ,tn }. Also, let A denote the allocation, A = {a1 , a2 , . . . , ak }, where

k 

ai = T , ai ⊆ T and the cluster ai is assigned to

i=1

agent hi . The cost associated with A is given by either k

C(A ) = ∑ c(ai )

(1)

C(A ) = max c(ai ),

(2)

i=1

or i

where c(ai ) is the minimum cost of agent i completing the set of tasks ai . The problem is to solve the optimization problem min C(A ). In practice the cost function A

in (1) might be used to represent the total distance traveled or the total energy expended by the agents, while the cost function in (2) might be used to represent the maximum time taken to accomplish the tasks.

2.2 Stochastic Clustering Auction (SCA) SCA attempts to minimize the cost C(A ) using a Markov chain search process in the space of possible allocations. The basic algorithm was originally developed in [10]. The algorithm is in the class called Markov Chain Monte Carlo [11]. The essential mechanism of SCA is to start with an allocation A for k clusters and to probabilistically reduce or hillclimb C(A ) by rearranging the tasks T among the clusters. The actual algorithm is described below. 2.2.1

Stochastic Clustering Auction

1. Partition the task set T into k clusters to form an initial allocation A = {a1, a2 , . . . , ak }, where each cluster ai is an unordered subset of T .

348

K. Zhang et al.

2. Each agent hi ∈ H (i = 1, 2, . . . , k) uses a “constrained Prim’s Algorithm”1 (a greedy algorithm) to efficiently approximate the cost ci (ai ) and submits its cost to the auctioneer. In this bid valuation stage, each cluster ai becomes an ordered subset of T . The auctioneer computes the global cost C(A ) using (1) or (2) and sets a high temperature T . 3. The auctioneer rearranges the clusters using either a single move or a dual move. a. Single Move: Randomly select a task ti ∈ aℓ from agent hℓ . Assume that ti ( j,ℓ) is reassigned to agent h j , resulting in the new allocation Ai that has two (−i) (+i) 2 (−i) . Assume that agent hl computes c(aℓ ) and modified clusters aℓ and a j (+i) 3 ) ,

for j = 1, 2, . . . , k agent h j computes c(a j

which the auctioneer uses to

( j,ℓ)

compute the corresponding cost C(Ai ) (based on (1) or (2)). The likelihood of the acceptance of the transfer of task ti from agent hℓ to agent h j is given by ( j,l) exp(−C(Ai )/T ) PS (i, j, k, T ) = k . (3) ( j,l) ∑ exp(−C(Ai )/T ) j=1

b. Dual Move (Task Swap): Randomly select two tasks in aℓ and am , task ti from agent hℓ and task t j from agent hm , and swap them, resulting in the new (ℓ,m) (−i,+ j) (+i,− j) 2 and am . Assume allocation Ai, j that has two modified clusters aℓ (−i,+ j)

that agent hℓ computes c(aℓ

(+i,− j) 3 ) ,

) and agent h j computes c(am

which

(ℓ,m) C(Ai, j ))

(based on the auctioneer uses to compute the corresponding cost (1) or (2)). Then, the likelihood of swapping the two tasks is given by PD (i, j, ℓ, m, T ) =

exp(−C(2) /T ) 2

∑

,

(4)

exp(−C(p) /T )

p=1 (ℓ,m)

where C(1) = C(A ), the cost before swapping, and C(2) = C(Ai, j cost after swapping.

)), the

4. If PS (i, j, k, T ) or PD (i, j, ℓ, m, T ) falls into acceptance likelihood, the auctioneer accepts the proposal so that A is updated and the cost C(A ) is put on log. If PS (i, j, k, T ) or PD (i, j, ℓ, m, T ) falls into rejection likelihood, the auctioneer declines the proposal and the auctioneer reserves the current configuration and goes back to Step 3. 5. If the auction evolution termination criteria is satisfied, i.e., T < Tcut , where Tcut is some threshold temperature, then the auction is terminated and the final 1 2 3

This algorithm fixes the initial vertex with a single edge in Prim’s Algorithm [12], and hence, unlike Prim’s algorithm, is not guaranteed tof be optimal. Each cluster is treated as an unordered subset and is ordered in a later bid valuation stage. This cost is computed using the constrained Prim’s algorithm during bid valuation stages.

A SCA for Centralized and Distributed Task Allocation in Multi-agent Teams ( j,l)

349

(ℓ,m)

allocation is A ∗ = Ai or A ∗ = Ai, j with final cost C(A )∗ . If the criteria is not satisfied, reduce T , using T ← T /β where β > 1 and go to Step 3. The above algorithm description only allows single moves and dual moves (task swapping). In the implementation of SCA used in this study, the algorithm alternated between single and dual moves. Simulation results (omitted for brevity) showed that when SCA alternates between single and dual moves it is more efficient than using exclusively single moves or dual moves. In order to search for the global optimal, a simulated annealing method has been adopted. Similar to the seminal paper in [13], SCA starts with a high value of T and gradually reduces it in order to to make small variations in the task allocation while searching for the optimal allocation in T . Although simulated annealing and random search help avoid local minimum, SCA does not guarantee convergence to a global optimum. It can be shown that if the temperature T is gradually decreased, then the algorithm will converge to a global optimum as described in [11], but it is difficult to quantify the rate of this decrease. Hence, the primary practical value of using simulate annealing is to yield nearly optimal solutions in reasonably fast execution times rather than guarantee asymptotic convergence results.

2.3 Distributed Task Allocation If all the mission tasks are given in T , then the SCA algorithm described above is a centralized auction. A centralized auction may make sense at the beginning of a mission, but as discussed in Section 1, due to limited communication and the computational cost of a centralized auction, it may not be feasible during the mission. Hence, once the mission begins, it is assumed that clustering must be performed in a distributed fashion in which each agent sequentially in a given (possibly random) order becomes the auctioneer. This approach of initializing a mission with a centralized auction and subsequently using a distributed auction is advocated in [6]. Assume that agent hi is the auctioneer for the set of heterogenous agents HL defined by HL = {h p : p ∈ L } where i ∈ L ⊂ {1, 2, . . . , k}. It follows that HL ⊂ H , and hence hi is the auctioneer for a distributed auction. If the initial global allocation (0) (0) (0) is given by A (0) = {a1 , a2 , . . . , ak }, then the set of tasks in the distributed auc(0)

tion is TD = {t ∈ a p : p ∈ L } ⊆ T . An allocation associated with the distributed auction is given by AL = {a p : p ∈ L } where the set of tasks in the allocation AL is TD , i.e.,TD = {t ∈ a p : p ∈ L }. Definition 1. The regional cost associated with the agents HL and the tasks TD is given by C(AL ) = ∑ c p (a p ) (5) p∈L

when the global cost C(A ) is defined by (1), and C(AL ) = max c p (a p ) p∈L

when the global cost C(A ) is defined by (2).

(6)

350

K. Zhang et al.

It can be proved that if the distributed auction is based on optimizing the regional cost, the global cost will either decrease or remain the same, which motivates basing distributed SCA on the optimization of regional costs. (The formal theorem statement and proof of this result are omitted for brevity.) The actual distributed auction using SCA can then proceed in a manner similar to the distributed auction described in [6]. In particular, each agent, sequentially or in random order, calls and clears one auction. Rounds are held repeatedly until a stable solution is reached. The auctioning process can recommence when a new task is obtained or when there is a substantial change in the existing costs.

3 Simulation Results This section provides simulation results for SCA using the multi-agent routing problem, which is a standard test domain for agent coordination using auctions [3, 5]. The tasks in the multi-agent routing problem considered here are to visit targets and complete an assignment at that target with exactly one agent assigned per target. The goal is to minimize the total time to overall task completion. In the simulations agent heterogeneity was taken into account by assuming that the agents moved at differing velocities and also differed in their completion times at each target. Sequential (single-item) Auctions (SA) and Parallel Auction (PA) populated in centralized task allocation are compared [1, 2, 3, 5, 6]. In SA unsigned tasks are auctioned off one in every round, and they are assigned to the winner promptly until all the targets are owned by robots [3]. In PA, every robot bids on each target simultaneously until all the targets are bided off to robots [3]. Centralized auctioning is first considered. Then, the use of SCA in a distributed auction is demonstrated. For each simulation unless otherwise specified the following SCA parameters were used: initial temperature, T = 100; cooling schedule rate, β = 1.001; and termination temperature, Tcut = 20. Random scenarios involving 2 agents and from 5 tasks to 50 tasks in increments of 5 were considered. For each number of tasks 20 random scenarios were generated based on randomly distributed task sites on a 1000 m × 1000 m area, random completion duration times for each agent at each task site in the interval [0, 2000] s and random constant speeds for each agent in the interval [0, 2] m/s. The initial robot positions are at [100 0] and [1000 0]. The cost function is a minmax cost function (2) corresponding to the mission completion time. SCA is evaluated for two cooling schedule ratios β . As β increases, the annealing process cools down more rapidly, leading to faster algorithm completion time but less optimal results. “CI” denotes the mean cost improvement of SCA for the 20 scenarios over the minimum of the SA and PA costs. “iterations”4 is the average number of auction evaluations. Note that due to the randomness of the distributions the mean costs do not monotonically increase as the number of tasks increase. Experiments (omitted for brevity) show that SCA with random initialization renders the lowest team cost of the auction methods evaluated; as expected, this is at 4

The “iterations” represent the bid evaluation cycle in Steps (3) and (4) of Section 2.2.1.

A SCA for Centralized and Distributed Task Allocation in Multi-agent Teams

351

(a) β = 1.001

(b) β = 1.01 Fig. 1 Mean Cost Improvement of SCA initialized with the better of SA or PA for β = 1.001 and β = 1.01

the expense of a longer runtime. However, the parameter β enables SCA to tradeoff optimality and computational performance. The performance of SCA with the parameter β = 1.01 enables it to render better performance than either SA or PA in relatively few (29 to 145) iterations. Figure 1 shows the average CI when the SCA is initialized with the better of the SA or PA solutions (in terms of global cost). It is seen in Figure 1(a) that

352

K. Zhang et al.

Fig. 2 Results of distributed task allocation via SCA for a benchmark problem (Each agent sequentially becomes the auctioneer as indicated by the circles. The regional communication is denoted by ↔)

initializations with SA or PA can reduce the iterations of SCA from the interval [290 1612] to [80 421], which results in a fourfold decrease in the communication load requirements. Furthermore, when the annealing suite is adjusted to speed up the convergence (β = 1.01) in Figure 1(b), an improvement of 6%-12% can be achieved on team performance in 8-35 iterations. Additional improvement can be achieved if the communication ability permits.

3.1 Distributed Auctions To illustrate the efficacy of distributed auctioning using SCA, a symmetrical benchmark is considered involving 4 homogeneous agents and 32 initial task sites arranged with communications available between neighboring agents so that the globally optimal solution is for each homogeneous agent to have an identical pattern of site visitation. SCA is used in a centralized auction to compute the optimal task allocation {a1 , a2 , a3 , a4 } in 1 iteration and a run time of 4.1 s. Subsequently, an additional task td is added during the mission (to the upper right corner of Figure 2(a).

A SCA for Centralized and Distributed Task Allocation in Multi-agent Teams

353

Figure 2 shows the ability of the distributed auction to yield optimal results5 . The 4 distributed auctions were performed sequentially and had the following computational performance: 1) 3 iterations with a 17.4 s runtime, 2) 3 iterations with a 21.3 s runtime, 3) 4 iterations with a 21.3 s runtime, and 4) 4 iterations with a 21.3 s runtime. These results indicate that if the number of changes in the tasks or mission is small, distributed auctioning using SCA may not require many iterations (i.e., auctions) to converge to an optimal or near-optimal solution.

4 Conclusion and Future Research In this paper we developed Stochastic Clustering Auction (SCA), a new auction algorithm based on the concept of stochastic clustering. SCA can be used for both centralized and decentralized auctions. When used for distributed auctions, it enables the auctioneer to allocate tasks to optimize the regional cost. For centralized auctioning a suite of random scenarios was used to compare SCA with the sequential (single-item) auction and the parallel auction. SCA can improve upon these methods at the expense of greater communication and computational requirements. A positive aspect of SCA is that the choice of the cooling parameter β enables a tradeoff between optimality and the the communication and computational requirements. In practice SCA can be used to improve the task allocation obtained by these less optimal but computationally simpler auction methods when the agents have access to sufficient communication. Hence, SCA provides a mechanism for continuous auction improvement. Future research will investigate mechanisms to speed up SCA and reduce the communication requirements for both central and distributed auctioning by only allowing certain tasks to be auctioned, e.g., those that have a high cost for the agent that has currently been assigned that task. SCA will also be extended to handle complex tasks [6], i.e., tasks that have interdependence. In addition, SCA will consider human risk by considering a constrained optimization problem (e.g., minimize total completion time subject to a constraint on human risk) and will consider human fatigue by allowing human agents to specify how many tasks and what kind of tasks they are willing to be assigned in the auctioning process. Experimental validation of these results is also underway. Acknowledgements. This work was sponsored, in part, by the US Army Research Laboratory under Cooperative Agreement W911NF-06-2-0041. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the ARL or the US Government. The US Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation hereon. The first author thanks Yiming Wu for helpful discussions. 5

It should be noted that the distributed auction will converge to the optimal solution no matter what sequence of auctioneers is chosen as long as it is a repeating sequence. However, alternative sequences may take more time or iterations to converge.

354

K. Zhang et al.

References 1. Gerkey, B.P., Mataric, M.J.: Sold!: auction methods for multirobot coordination. IEEE Transactions on Robotics and Automation 18(5), 758–768 (2002) 2. Simmons, R., Singh, S., Heger, F., Hiatt, L.M., Koterba, S.C., Melchior, N., Sellner, B.P.: Human-robot teams for large-scale assembly. In: Proceedings of the NASA Science Technology Conference 2007 (NSTC 2007) (May 2007) 3. Dias, M.B., Zlot, R.M., Kalra, N., Stentz, A.: Market-based multirobot coordination: a survey and analysis. Proceedings of the IEEE 94(7), 1257–1270 (2006) 4. Koes, M., Sycara, K., Nourbakhsh, I.: A constraint optimization framework for fractured robot teams. In: Proceedings of the Fifth International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS), pp. 491–493 (May 2006) 5. Koenig, S., Tovey, C.A., Zheng, X., Sungur, I.: Sequential bundle-bid single-sale auction algorithms for decentralized control. In: IJCAI, pp. 1359–1365 (2007) 6. Zlot, R.M., Stentz, A.: Market-based multirobot coordination for complex tasks. Int. J. of Robot. Res., Special Issue on the 4th International Conference on Field and Service Robotics 25(1), 73–101 (2006) 7. Parker, L.E.: Alliance: An architecture for fault tolerant multi-robot cooperation. IEEE Transactions on Robotics and Automation 14(2) (1998) 8. Wagner, A., Arkin, R.C.: Multi-robot communication-sensitive reconnaissance. In: Proceedings of the IEEE International Conference on Robotics and Automation, New Orleans, LA, April 26 - May 1, pp. 4674–4681 (2004) 9. Sandholm, T.: An algorithm for optimal winner determination in combinatorial auctions. In: IJCAI 1999, pp. 542–547 (1999) 10. Srivastava, A., Joshi, S.H., Mio, W., Liu, X.: Statistical shape analysis: Clustering, learning, and testing. IEEE T. on Pattern Anal. 27(4), 590–602 (2005) 11. Robert, C.P., Casella, G.: Monte Carlo Statistical Methods (Springer Texts in Statistics). Springer, New York (2005) 12. Prim, R.C.: Shortest connection networks and some generalisations. Bell System Technical Journal 36, 1389–1401 (1957) 13. Kirkpatrick Jr., S., Gelatt, C.D., Vecchi, M.P.: Optimization by simulated annealing. Science 220(4598), 671–680 (1983)

Corridors for Robot Team Navigation Zack Butler and Carlos Bribiescas

Abstract. In previous work, we proposed the use of virtual corridors to move a group of agents (animals) from one location to another. Using virtual fence technology, we can create a corridor from a set of fences to generate the desired gross motion of the herd without overspecifying paths for the animals. This allows us to reduce the amount of stimulus given to the animals while allowing them to naturally avoid obstacles (or other agents) without having to consider them during planning. In addition, a single corridor can be used by the entire group, simplifying planning and requiring only limited communication. In this work, we have implemented virtual corridors for robot team navigation, both in simulation and on two Pioneer mobile robots, and performed a number of experiments with two aims in mind. First, we wish to determine if, and under what circumstances, corridors are a practical solution to navigation by a team of robots. Secondly, by investigating different underlying motion algorithms for the robots, we may be able to suggest how successful corridors can be for different systems (including animals). We show success in open environments over a variety of experiments, with varying efficiency from the different motion models, and partial success among complex obstacles and with teams of up to five robots using a single corridor simultaneously.

1 Introduction Unlike robots, animals have innate navigational and obstacle-avoidance capabilities. However, unlike animals, robots will (generally) go where they are told. In our past work [3], we developed the idea of virtual fences to bring Zack Butler and Carlos Bribiescas Dept. of Computer Science Rochester Institute of Technology Rochester NY USA e-mail: [email protected],[email protected]

356

Z. Butler and C. Bribiescas

some level of path planning and control to the animal systems. That is, by putting a device on the animal’s collar which can detect the animal’s position, we can define a virtual fence so that the collar induces a stimulus on the animal when it crosses this fence, thus directing the animal to remain in a predefined area. The positive effect of such stimuli has been verified in [7]. In addition, since the collar device includes both GPS and computation, it is easy to make the virtual fences move over time to move the animals as desired. Finally, it is also possible to use the same set of virtual fences for all the animals in a herd, greatly simplifying the planning for this cooperative system, leveraging the animals’ local navigation capabilities and inducing stimuli only to achieve global path goals. In this work, we are interested in looking at the problem from a robotics viewpoint. That is, can robots with simple obstacle-avoidance capability use the same type of virtual fences to induce path following in a simple way? This would provide several avenues of simplification: a single plan can be generated and used for a team of robots to get from one area to another, dynamic obstacles may be implicitly handled within the corridor, and the robots do not need to able to communicate with or explicitly recognize each other to execute the path. We are also interested in the level of innate navigational ability required to successfully follow the desired path — is a random walk sufficient, or are maps or state-based planning necessary? Our technique requires a base map to develop the corridors, but the width of the corridors allows the robots to deal with small unmapped obstacles and dynamism of the environment with no additional overhead. A large amount of work has been done in cooperative robotics in terms of multi-robot path planning. This can take the form of a single plan for all robots, essentially forming a plan in a 2n dimensional space for n robots that handles collision avoidance by adding collisions as obstacles in the configuration space (e.g. [9]). Alternately, each robot can make a plan independently, with distributed traffic control used to prevent collision and deadlock (e.g. [5]). These plans can handle the case of robots with qualitatively different goal locations, but may not be suited to dynamic environments as we expect to encounter here. For robots traveling together, formation control can be used, in which a leader robot has a path to follow and the others take a position relative to the leader. With appropriate control policies, this has been shown to be robust even in the presence of obstacles that force the robots out of formation [1, 4]. However, the role of each robot within the formation must be specified and possibly reassigned in the case of failure. In the algorithm used here, a single plan used is homogeneously across all robots in the system. Explicit inter-robot sensing and/or communication can also be used to improve navigation during path execution [8]. In contrast, our work does not rely on such sensing, and explores what can be accomplished when other robots are treated as (dynamic) obstacles within a single planned path.

Corridors for Robot Team Navigation

357

(a)

(b)

(c)

(d)

Fig. 1 Example of corridor creation. (a) A path is planned for a single point. (b) The path segments are extended in one direction to form corridor walls, going through an obstacle in the environment. (c) The expansion on the other side is stopped to avoid creating a dead end in the corridor. (d) If an obstacle is only algorithmic, that is, the corridor must keep the agents out of the given area, this can also be handled during the expansion as shown

1.1 Corridor Planning Using dynamic virtual fences, we have previously developed a planning algorithm [2] that creates a virtual corridor through the environment to herd a group of agents from one given area to another. The underlying motivation is twofold. First, that having the majority of the virtual fences fixed over time (as opposed to a moving rectangle that contains the group) is both simpler to create and it would seem more intuitive for the agents to follow. Second, the corridors can be embedded in the natural obstacles of the environment, so that where possible, the visible features are used to control the agents rather than invisible ones. This latter concept also can allow corridors with simpler shape as they can go straight through obstacles. The process to create a corridor in a sample environment is given in Fig. 1. First, a path is first created for a single point using a straightforward A* implementation. This is initially limited to travel along the eight cardinal directions, and the path costs used by A* are chosen such that the path will have as few turns as possible, so that the resulting corridors will have fewer segments. Each segment of this point-based path is then moved perpendicularly in both directions to create a corridor segment. Care is taken to avoid introducing local dead-ends in the corridor. We can also specify areas which the corridor must not contain (such as a set of stairs that the robot cannot sense), and the expansion process will simply stop at those points. We then

358

Z. Butler and C. Bribiescas

(a)

(b)

Fig. 2 Block diagram of collar architecture. (a) In the animal system and (b) in an indoor robot system. In both cases, external control communicates with onboard computing intermittently through a wireless connection (dashed lines), while fencebased control takes place in real time on the collar (solid lines). In the robot, both robot control (random walk) and collar control operate on the same computer, but in separate threads

create a single moving wall that travels along the length of the corridor to keep the agents moving toward the goal area. The moving wall is oriented perpendicularly to the original path for each segment except at corners, where it pivots around the vertex at the inside of the corner. For a full description of this technique, see [2].

2 System Architecture In the animal implementation of the virtual corridors, each animal is given a collar with GPS, computation, wireless communication and a stimulus device. At user-defined intervals, a set of virtual fences is uploaded to the animals, and these fences are then enforced by the computer via the stimulus device. It is usually the case that all the animals in a group will get the same fences, though this is not required. The fences may move according to a schedule to enable mustering from one location to another, or may simply be fixed as normal fences. A block diagram of this system can be seen in Fig. 2a. To implement this on robots, we simply replace the animal with a robot that is programmed to move about and react to “stimulus” (whatever that may mean in the context of the implementation). We have implemented this with a multi-threaded client program in the Player/Stage framework [6]. One thread runs simple motion control that communicates via the Player interface with the robot hardware (or Stage simulation). A second thread runs the collar control program which obtains fences from a data file. A shared data object allows for stimuli to be sent from the collar to the robot. For the case of indoor mobile robots, we must also find an alternative to GPS. For simplicity in this case, as seen in Fig. 2b, we have used dead reckoning, with another shared data object sending robot positions from the motion

Corridors for Robot Team Navigation

359

control thread to the collar thread. We note that since Player/Stage simulates perfect dead reckoning, the simulation is more equivalent from a functionality standpoint to the GPS-based outdoor case than the indoor case even though the implementation is the same as is used indoors.

3 Motion Models As mentioned previously, an important facet of this work is the evaluation of different underlying robot motion behaviors for use with the virtual corridors. These include simple random walks with varying parameters as well as the use of wandering with additional state. The simplest motion model used is a safe random walk. In this model, the robot first checks that no obstacle is within approximately one robot diameter ahead of it, and if such an obstacle is present, it stops forward progress and turns toward the side with greater clearance. Otherwise, it uses its four forward-facing sonar sensors (covering approximately ± 30◦ ) to set its forward velocity. It averages the distances of these sensors and divides the average distance by a speed factor (essentially setting a time-to-collision). For our robots, the sonar readings are capped at 5 m, so a speed factor of 10 corresponds to a maximum velocity of 0.5 m/s. It then chooses a random angular velocity and executes the motion. In these experiments we use a uniformly random angular velocity between -0.5 and 0.5 rad/s, though we have also tested other models with similar results. In addition to the unconstrained motion, there are different ways in which the robot can react to a stimulus, even within the domain of attempting to model likely animal behavior. Inspired by the animal hardware, the stimulus itself is simply bilateral, that is, it can either be on the right or left of the robot but with no additional direction or magnitude information. The robot will then react to the stimulus by turning away from it by a predetermined amount — that is, if the robot drives forward into a right-side virtual fence, it will receive a stimulus on the right, and thus turn toward the left. In this work, we have used responses of a constant turn of 90◦ , a constant turn of 45◦ , and a uniformly random turn between 45◦ and 135◦ . Going up a step in complexity, the robots can keep some state about the stimuli they have received. As a first approach, we have chosen to keep a “drift” vector d that remembers in what direction stimuli has been received and decays over time. Each time a stimulus is felt, the robot computes an aversion direction perpendicular to its current direction of travel and adds this aversion (with magnitude 0.25) to d. The magnitude of d is capped at 0.5 to ensure stable behavior. Each time step that no stimulus is felt, the robot computes a random vector r with magnitude 0.5 based on the random walk used above, and the vector d + r is multiplied by the nominal speed to determine the velocity of travel (note that the magnitude limits mean that the nominal speed will never be exceeded). In addition, the magnitude d is

360

Z. Butler and C. Bribiescas

decreased by .0075 per second to simulate the slow forgetting of stimuli in the past (and allow for virtual walls in different directions to be handled more effectively).

4 Experimental Results We ran a number of experiments in different environments to attempt to determine the utility of corridor planning. These included varying the motion model, both qualitatively and quantitatively, and the speed of the back wall of the corridor. The majority of the experiments were done with two robots sharing a single corridor, though we also performed some tests with five robots to explore the parallelism available. We defined success as whether the robots reached the last 2 meters (of the > 25 m corridor) before the back wall reached the end of the corridor. The first set of experiments was designed to determine which stimulus response would be most effective in the context of the corridors. We tested a straight empty virtual corridor with the velocity of the moving back fence (vf ) between 0.02 m/s and 0.1 m/s and robot speed factors (sf ) of 10 and 20 (recall that since this number divides the sonar reading to obtain the forward velocity, a larger sf indicates slower nominal velocity). Over all the stimulus responses, we found that the fence velocity had only a moderate effect on the time taken to travel the corridor, but a larger effect on the frequency of successful completion of the corridor. For the 90◦ response, for example, the robots were able to travel the corridor in all trials with sf = 10 and vf = 0.02 and 0.04, and in these two cases, the time taken to reach the end of the corridor was 199.5 s and 189.5 s respectively. As the fence velocity increased, the success rate decreased, with 16/20 trials succeeding at vf = 0.08 and only 5/20 at vf = 0.1. In this last case, when the robots did succeed, they did so in an average of 138.7 s, somewhat faster than with the slower fence, as expected. The random-turn stimulus response had similar rates of success at the higher robot velocity, but with significantly longer time taken to travel the corridor (294 s at vf = 0.02 and 301 s at vf = 0.04). At the slower robot velocity and/or higher fence velocity, the success rate was significantly lower than the 90◦ response, with only 2/20 completing at vf = 0.1 and sf = 10. Finally, for the 45◦ response, we found higher success rates (100% success at fv = 0.08, sf = 10 and vf = 0.04, sf = 20) and faster traversal times (156.1 s at fv = 0.02 and 144.8 s at fv = 0.04) than either of the other responses. Our observations are that the long and narrow aspect of the corridor rewards smaller turns that tend to point the robot along the corridor rather than back across it. We also analyzed the number of stimuli that had to be applied to get the robots to reach the end of the corridor in each case. Here we found correlation between the amount of stimulus and the time to completion. For the 90◦ response and the 45◦ response, stimulus levels were quite similar, averaging

Corridors for Robot Team Navigation

(a)

(b)

361

(c)

Fig. 3 Environments used in simulation experiments: (a) “L” test (b) “Square” test (c) “Complex” test. Solid shapes represent physical obstacles and dashed lines represent virtual fences. The arrow indicates the moving back wall of the corridor (shown at its initial configuration. Robots start in a group near the back wall. All environments are between 25 and 30 meters in width

approximately 20 total stimuli appiled to the two robots for the velocities with 100% success, while the random-angle response required significantly more stimulus, approximately 40-45 per run under the same conditions. Using the 45◦ stimulus response with sf fixed at 10, we then tested a variety of other environments to see how well it can move teams through areas of greater complexity. These environments are shown in Fig. 3. On the “L” test, the robots completed the corridor in 100% of tests up to fv = 0.06, and 90% of tests at fv = 0.08 and 0.1. Stimulus levels were actually lower than those in the blank corridor, since the real wall acts as one side of the corridor and can be avoided without external stimulus. The “square” and “complex” tests, on the other hand, are somewhat more challenging. Depending on the heading of the robot when it hits the moving fence, the stimulus may direct it toward an obstacle, and the robot will not be able to escape both the virtual and physical walls. This is especially pronounced in the complex test, where the moving back wall meets some obstacles at an acute angle. Also, even though the robots are fast enough to outrun the back wall, around the corners of obstacles, when the back wall turns to follow the direction of the corridor, it may actually provide a stimulus toward the previous segment of the corridor depending on the robot’s heading. We also saw occasional livelock when the two robots would meet at certain angles near a wall due to the simple obstacle avoidance. In the square tests, we obtained success rates of about 80-90% for all fence velocities, while in the complex test, there was a more pronounced dependence on fv . At fv = 0.02, 9 of 21 trials succeeded, though in another 8 trials, one of the two robots made it to the end of the corridor. At fv = 0.04, 7 of 21 trials were successful, but only 3 at fv = 0.06. In all case, in the trials that did succeed, stimulus levels remained low, since a large fraction of each corridor is defined by physical features.

362

Z. Butler and C. Bribiescas

We also tested five robots in the some of the same corridors. In these cases, the implicit traffic control gets a much more difficult test, as the virtual corridor is less than 10 robot widths across. As such, the success rates go down and the traversal times increase compared to the two-robot case. However, the algorithm still produces good results. In the blank corridor, an average of at least 4.8 out of the 5 robots reached the end in the allotted time for all fence speeds up to 0.08 m/s, though with completion times ranging from 362 s to 489 s, approximately three times as long as two robots in the same size corridor. The amount of stimulus required was slightly higher even on a per-robot basis, averaging between 61.3 and 77 total events per run. Similarly, the “L” test took three to four times as long as the two-robot case and with 4.5 robots or better finishing the corridor at fv = 0.02 and 0.04. In the “complex” test, with narrow physical as well as virtual corridors, an average of 3.5 and a minimum of 2 of the 5 robots finished each trial at fv = 0.01. Finally, we tested the drift-based control in the same environments to judge whether maintaining additional state can improve the efficiency of using the corridors. We retained the 45◦ stimulus response for direct comparison to the stateless model. For the blank corridor, we found significant performance improvements. Specifically, the robots were 100% successful at even higher fence speeds (up to fv = 0.1 at sf = 10 and fv = 0.06 at sf = 20) than any of the previous motion models. These are achieved with approximately half the number of stimuli used in the stateless case, and with slightly lower traversal times. With five robots in this corridor, similar success rates are seen as with the stateless case, but again at higher vf , with an average of at least 4.5 robots reaching the end of the corridor up to vf = 0.1. In contrast, the success rates in the other corridors were significantly lower than with the stateless technique. In the square corridor, 90% of trials succeeded at fv = 0.02, but only about 50% at fv = 0.04 and 0.06. In the complex corridor, only about 30% of the time did even one of the two robots finish. Our observations were that even with the decaying drift vector, when the robot came to the corner of two virtual fences, it would often travel directly into the far fence, and end up with a drift parallel to the corridor wall. The drift also exacerbated the problem of the virtual wall conflicting with the physical wall, as the effect of the stimulus against the physical wall would linger.

4.1 Hardware Experiments To test the viability of the algorithm in hardware, we implemented corridor following on our two Pioneer P3-DX robots. Each robot has eight forwardfacing sonar sensors as in our simulations, and only dead reckoning was used for positioning, as in Fig. 2b. Two environments were tested, one in open space and the other along a hallway. In the first set of experiments, a virtual corridor of length 15 m and width 1.8 m was used within the atrium of our building. There were no permanent

Corridors for Robot Team Navigation

363

obstacles within the virtual corridor. We ran five trials in which the two robots were started at the same time and allowed to run down the corridor. In all cases, both robots successfully reached the other end of the atrium. Dead reckoning error meant that their actual positions were occasionally outside the intended corridor, but never by far enough to cause collisions or confusion on the part of the robots. Additionally, because of the nature of the algorithm, occasional intrusions into the corridor by passers-by were also handled successfully without replanning. The second set of experiments was run in a hallway of width 2.1 m and length 20 m in which the virtual corridor was wider than the hallway. Thus, only the moving back wall was used to herd the robots down the hall. In one of the five experiments run, both robots were able to reach the end of the hallway. In the other four, one of the robots was able to travel the entire hallway, while the other did not. The effect looked to be similar to the simulations in which the back wall and physical wall give contradictory forces, however in this case it also seemed that the limited angular acceleration of the hardware caused a slower stimulus response than in simulation and made it more difficult to escape the back wall once it caught up to the robot.

5 Conclusions Overall, the work here has shown that the current implementation of virtual corridors can be used for path execution simultaneously by a team of robots, both simulated and in hardware, with greatest success in open environments. We found significant differences in the results with different stimulus response behaviors, with a smaller turn being more effective than a larger or random turn. Using the most effective behavior, the robots are able to avoid each other and small obstacles while reaching the end of the corridor. In simulated environments without obstacles (with respect to the virtual corridor), the robots completed their mission every time as long as the fence speed was less than 15% of the nominal robot speed, and environments with few obstacles, even large ones, could be traversed most of the time. The most complex environment tested could be handled with moderate success. Adding a small amount of state to the motion control improved the success rates and efficiency in straight corridors, but decreased performance in complex environments. Future work will entail investigating other ways to incorporate stimuli into memory to improve efficiency in more environments, especially to handle situations where the virtual fences and physical obstacles give contradictory impulses. We are also working on extending the control algorithm to include herding behaviors, that is, robots will be attracted to each other in the same way that herd animals are by nature. This will require some type of mutual sensing, or at least communication of positions, but may allow easier recovery from leaving the corridor or falling behind the moving fence as the robot outside

364

Z. Butler and C. Bribiescas

the corridor will see the other robots still within it. Other animal traits such as leading and following can also be tested to judge their effects (either positive or negative) under corridor control. On the other hand, to make this algorithm more suitable to the pure robotics context, we can alter both the corridor execution as well as the control. On the corridor side, we may be able to improve performance in complex environments and with the hardware (at the expense of execution speed) by using a ratcheting back wall instead of the constant-speed wall. That is, the back wall will move up as far as the last robot in the corridor, but not beyond, so that this robot can catch up to the group. This will also require communication between robots to keep the back wall the same within the herd, and may be less necessary if the herding behavior is used. Without adding communication, we can still fold in simple localization techniques and test these on the indoor hardware. In the obstacle-rich indoor environment, we may be able to specify virtual fences with respect to known physical obstacles, thus circumventing the localization problem. This would be similar to topological path planning and execution, but with the added benefit of shared corridors for efficient team navigation without explicit mutual sensing. Acknowledgements. This work was supported in part by NSF grant IIS-0513628.

References 1. Bicho, E., Monteiro, S.: Formation control for multiple mobile robots: a nonlinear attractor dynamics approach. In: Proc. of IEEE IROS, pp. 2016–2022 (2003) 2. Butler, Z.: Corridor planning for natural agents. In: IEEE International Conference on Robotics and Automation (2006) 3. Butler, Z., Corke, P., Peterson, R., Rus, D.: From robots to animals: Virtual fences for controlling cattle. Int’l. Journal of Robotics Research 25(5), 485–508 (2006) 4. Desai, J., Ostrowski, J., Kumar, V.: Modeling and control of formations of nonholonomic mobile robots. IEEE Trans. on Robotics and Automation 17(6), 905– 908 (2001) 5. Grossman, D.D.: Traffi c c ontrol of multiple robot vehicles. IEEE Journal of Robotics and Automation 4(5), 491–497 (1988) 6. The player project, http://playerstage.sourceforge.net 7. Quigley, T.M., Sanderson, H.R., Tiedemann, A.R., McInnis, M.K.: Livestock control with electrical and audio stimulation. Rangelands, pp. 152–155 (June 1990) 8. Sanderson, A.: A distributed algorithm for cooperative navigation among multiple mobile robots. Advanced Robotics 12(4), 335–349 (1998) 9. Whitcomb, L., Koditschek, D., Cabrera, J.: Toward the automatic control of robot assembly tasks via potential functions: The case of 2-d sphere assemblies. In: Proc. of IEEE ICRA, pp. 2186–2191 (1992)

Part VI

Practical Control of Modular Robots

Efficient Distributed Reinforcement Learning through Agreement Paulina Varshavskaya, Leslie Pack Kaelbling, and Daniela Rus

Abstract. Distributed robotic systems can benefit from automatic controller design and online adaptation by reinforcement learning (RL), but often suffer from the limitations of partial observability. In this paper, we address the twin problems of limited local experience and locally observed but not necessarily telling reward signals encountered in such systems. We combine direct search in policy space with an agreement algorithm to efficiently exchange local rewards and experience among agents. We demonstrate improved learning ability on the locomotion problem for self-reconfiguring modular robots in simulation, and show that a fully distributed implementation can learn good policies just as fast as the centralized implementation. Our results suggest that prior work on centralized RL algorithms for modular robots may be made effective in practice through the application of agreement algorithms. This approach could be fruitful in many cooperative situations, whenever robots need to learn similar behaviors, but have access only to local information.

1 Motivation Distributed robotics systems, such as teams of vehicles, robotic swarms, or modular robots, where each individual agent has to engage in decision-making under uncertainty and coordinate their activity, present a challenge in controller design. Also, they need to be robust and adaptable to changing environments. Both these challenges can be addressed by the application of reinforcement learning (RL) algorithms. However, often only the most straightforward RL techniques such as Q-learning are applied to groups of robots (e.g., [9, 4]), techniques that rely on the Markov property of the world. Distributed robotic systems, on the other hand, have limitations that violate the Markov assumption: individual agents do not have Paulina Varshavskaya, Leslie Pack Kaelbling, and Daniela Rus Massachusetts Institute of Technology, Computer Science and Artificial Intelligence Laboratory, Cambridge MA, USA e-mail: {paulina,lpk,rus}@csail.mit.edu

368

P. Varshavskaya et al.

access to the full state of the world, but only a partial observational window onto it, as delimited by their sensors. In fact, the process is even nonstationary if several agents learn, i.e., change their behavior, at the same time. This partial observability means that more general RL algorithms need to be used in distributed robotics, such as methods based on belief-propagation or methods based on direct search in policy space (e.g.,[1, 12]). The latter have been applied in particular to self-reconfiguring modular robots (SRMRs); the approach [15] proved fruitful in the case of distributed controller generation with a learning algorithm, when the learning was done centrally by a single “agent” from the experience gathered by individual modules with all their observational limitations. However, the fully distributed implementation of this work, where each module learns its own policy from its own local experience and reward only, has not been immediately successful and has required extra heuristics to work even for relatively small SRMR systems [16]. In fact, the observation that individual agents have access only to their own local experience and reward signal goes against a common assumption in the cooperative distributed RL literature (e.g., [12, 3]) that the reward is a global scalar available unaltered to all agents1 . Instead, the problem for locally sensing SRMR modules or robotic agents in a group is that the local reward is not necessarily a good measure of the agent’s performance. For instance, it is possible that the agent’s policy is optimal, but the bad choices made by others prevent it from ever finding itself in a configuration from which it can move. We address here the twin problems of limited local experience and locally observed but not necessarily telling reward signals through the application of agreement algorithms [2]. We demonstrate below that agreement can be used to exchange both local reward and local experience among agents, and will result in a fully distributed implementation of a direct policy search learning algorithm, which learns good policies just as fast as the centralized implementation2. In what follows, we first give a concise background on reinforcement learning by policy search in Sect. 2. We then incorporate agreement algorithms into the learning process in Sect. 3. Section 4 gives experimental results on a simulated distributed system: a self-reconfiguring modular robot. We discuss our contribution in light of related work in Sect. 5 and give directions for future research in Sect. 6.

2 Distributed Policy Search Suppose a group of robots runs a distributed policy search algorithm in the episodic reinforcement learning framework: at each timestep t each robot i observes the situation inside its local sensory and communications range (observation oti ) and chooses an action ati according to its own policy π i . At the end of the episode, each robot 1 2

This assumption is not made in general-payoff fully observable stochastic Markov games [6]. The centralized implementation, which serves as a benchmark, can be viewed as an idealized case where all modules have access to all the experience and rewards of all others.

Efficient Distributed Reinforcement Learning through Agreement

369

Algorithm 1. Distributed GAPS (DGAPS) for robot i (observation function o) initialize parameters θi ← small random numbers for each episode do calculate policy π (θi ) initialize observation counts N ← 0 and observation-action counts C ← 0 for each timestep in episode do observe oi and increment N(oi ) choose ai from π (oi , θi ), increment C(oi , ai ) and execute ai end for get local reward Ri and update θi ( oi , ai ) + = η Ri (C(oi , ai ) − π (oi , ai , θi ) N(oi ) ) update π (θi ) using Boltzmann’s law end for

receives a scalar reward signal Ri , which is derived from the robot’s local measurements. This process is formalized as a distributed Partially Observable Markov Decision Process (POMDP). The basic Gradient Ascent in Policy Space (GAPS) algorithm [12] does hillclimbing to maximize the value (that is, long-term expected reward) of the parameterized policy. The derivation starts with noting that the value of a policy πθ is Vθ = Eθ [R(h)] = ∑hε H R(h)P(h|θ ), where θ is the parameter vector defining the policy, R is the reward function, and H is the set of all possible experience histories. If we could calculate the derivative of Vθ with respect to each parameter, it would be possible to do exact gradient ascent on the value by making updates ∆ θk = α ∂∂θ Vθ . k However, we do not have a model of the world that would give us P(h|θ ) and so we will use stochastic gradient ascent instead. The partial derivative of the value of a policy with respect to one policy parameter is (see [12] for full derivation)   T ∂ ∂ V = R(h) P(h|θ ) ∑ ln πθ (at , ot ) . ∂ θk θ h∑ t=1 ∂ θk εH

(1)

The stochastic gradient ascent algorithm operates by summing up traces of logpolicy derivatives at every timestep of each learning episode. Each trace reflects the contribution of a single parameter to the estimated gradient. The makeup of the traces will depend on the policy representation. The GAPS algorithm was originally derived [12] for a lookup-table representation, where there is a parameter θ (o, a) for each observation-action pair. The fully distributed version of GAPS (DGAPS) is reproduced here for completeness (Alg. 1). The traces are obtained by counting occurrences of each observation-action pair, and taking the difference between that number and the expected number that comes from the current policy. The policy itself, i.e., the probability of taking an action at at time t given the observation ot and current parameters θ is given by Boltzmann’s law: πθ (at , ot ) = P(at |ot , θ ) =

eβ θ (ot ,at )

∑a∈A eβ θ( ot ,a)

,

(2)

where β is the inverse temperature. This gradient ascent algorithm has some important properties. As with any stochastic hill-climbing method, it can only be relied upon to reach a local optimum

370

P. Varshavskaya et al.

in the represented space, and will only converge if the learning rate is reduced over time. Furthermore, distributed GAPS will converge to the same local optimum as its centralized version (theorem 3.1 in [12]) provided individual learning robots get the same experience and reward as would the central learner in the centralized case. Clearly, this condition is not satisfied in any distributed robot system where individuals have only access to their own local observations (which may well be different from anybody else’s) and compute rewards based on local information. To counteract that limitation, we introduce agreement algorithms into the learning process.

3 Agreement Algorithms in Policy Search Agreement (also known as consensus or flocking) algorithms are pervasive in distributed systems research. First introduced by Tsitsiklis et al.[14], they have been employed in many fields including control theory, sensor networks, biological modeling, and economics. They are applicable in both synchronous and partially asynchronous settings.

3.1 Basic Agreement Algorithm Suppose there are N robots and therefore N processors, where each processor i is updating a variable xi . For ease of exposition we assume that xi is a scalar, but the results will hold for vectors as well. Suppose further that each i sends to a subset of others its current value of xi at certain times. Each module updates its value according to a convex combination of its own and other modules’ values: N

xi (t + 1) =

∑ ai j x j (τ ij (t))

if t ∈ T i ,

xi (t + 1) = xi (t) otherwise,

(3)

j=1

where ai j are nonnegative coefficients which sum to 1, T i is the set of times at which processor i updates its value, and τ ij (t) determines the amount of time by which the value x j is outdated. If the graph representing the communication network among the processors is connected, and if there is a finite upper bound on communication delays between processors, then the values xi will exponentially converge to a common intermediate value x such that xmin (0) ≤ x ≤ xmax (0) (part of Proposition 3.1 in [2]). The agreed-upon value x will depend on the particulars of the averaging process. It will be equal to the exact arithmetic mean of the initial values xi if the processors make synchronous, pairwise disjoint, symmetrical updates (under sum invariance [2]): xi (t + 1) = x j (t + 1) =

 1 xi (t) + x j (t) . 2

(4)

Agreement algorithms can also be used for gradient-style updates performed by several processors simultaneously in a distributed way [14]. However, GAPS is a stochastic gradient ascent algorithm, which means that the updates performed climb the estimated gradient of the policy value function ∇Vˆ , which may, depending on the quality of the agents’ experience, be very different from ∇V , and may not satisfy

Efficient Distributed Reinforcement Learning through Agreement

371

Algorithm 2. Synchronous average-based collection of experience. This algorithm may be run at the end of each episode, or the procedure labeled average may be used at each time step while gathering experience. send to all n neighbors message = R,Co ,Coa  for all time steps do average: m receive from all n neighbors m = Rm ,Com ,Coa  message 1 n m average reward R ← n+1 R + ∑m=1 R     1 1 m average experience Co ← n+1 Co + ∑nm=1 Com , Coa ← n+1 Coa + ∑nm=1 Coa send to all n neighbors m′ = R,Co ,Coa  end for update: GAPS update rule

the assumptions of gradient optimization with agreement. In addition, there may be correlated noise in a system where agents must maintain physical or wireless connections.

3.2 Agreeing on Common Rewards and Experience Since agreement is a distributed (weighted) averaging procedure, it is immediately applicable to learning scenarios where the cooperative reward R can be expressed as a (weighted) average of the reward signals received by individual learning agents. We expect the agreed-upon Rˆ to be a more accurate measure of how well the entire system is doing than the individual local reward Ri , which will increase the likelihood of better updates by all robots. The estimate will be exactly right: Rˆ = R if the robots use synchronous, symmetrical, pairwise disjoint updates (Eq. 4), and the original R is the arithmetic mean of local Ri ’s, such as when SRMR modules learn a locomotion policy [16]. A theorem in [12] states that a distributed version of GAPS, where each agent maintains and updates its own policy parameters, will make exactly the same updates as the centralized version of GAPS, provided the same experience and the same rewards. In addition to sharing local rewards, what happens if modules also exchange local experience? In the centralized GAPS algorithm, the parameter updates are computed with the following rule: ∆ θoa = α R (Coa − Co πθ (o, a)), where the counters of experienced observations Co and chosen actions per observation Coa represent the sum of the agents’ individual counters. If there is a way for the agents to obtain these sums through local exchanges, then the assumptions of the theorem are satisfied and we can expect individuals to make the same updates as the global learner in the centralized case, and therefore to converge to a local optimum. Recall that using synchronous, equal and symmetric pairwise disjoint updates guarantees convergence to a common x which is the exact average of initial values of xi . If at the end of an episode, each module runs such an agreement algorithm on each of the counters that

372

P. Varshavskaya et al.

(a)

(b)

(c)

Fig. 1 The sliding cubes model for lattice-based reconfiguration: (a) sliding and (b) convex transitions. (c) The abstract kinematic model of a SRMR on a lattice in 2D

it maintains, then each counter Ci will converge to C = N1 ∑Ni=1 Ci . If this quantity is then substituted into the GAPS update rule, the updates become: ∆ θoa = α R



1 N

N

∑

i=1

i Coa − πθ (o, a)

1 N

N

∑

i=1

Coi



=

1 α R (Coa −Co πθ (o, a)) , N

(5)

which is equal to the centralized GAPS updates scaled by a constant N1 . Note that synchronicity is also required at the moment of the updates, i.e., all agents must make updates simultaneously, in order to preserve the stationarity of the underlying process. Therefore, robots learning with distributed GAPS using an agreement algorithm with synchronous pairwise disjoint updates to come to a consensus on both the value of the reward, and the average of experience counters, will make stochastic gradient ascent updates, and therefore will converge to a local optimum in policy space. The algorithm (Alg. 2) requires the scaling of the learning rate by a constant proportional to the number of modules. Since stochastic gradient ascent is in general sensitive to the learning rate, this pitfall cannot be avoided. In practice, however, scaling the learning rate up by an approximate factor loosely dependent (to the order of magnitude) on the number of agents in the system, works just as well.

4 Experiments We ran the synchronized algorithm described as Algorithm 2 on a case study task of locomotion by self-reconfiguration in 2D, following prior work on GAPS for lattice-based SRMRs [16].

4.1 Lattice-Based Self-reconfiguring Modular Robots We use the abstract kinematic model of a 2D lattice-based robot as shown in Fig. 1. The modules are constrained to be connected to each other in order to move with respect to each other; they are unable to move on their own. The robot is positioned

smoothed average displacement

18 16 14 12

basic DGAPS (1) with given mean R (2) with agreement on reward at every t (3) with agreement on reward at end of episode (4)

4

average displacement

Efficient Distributed Reinforcement Learning through Agreement

2 3

10 8 6 4

1

15 3 basic DGAPS (1) agreement with original learning rate (2) agreement with scaled learnig rate (3)

2

10

5 1

2 0 −2 0

373

0 20,000

40,000 60,000 episode

(a)

80,000

100,000

0

20,000

40,000 60,000 episode

80,000

100,000

(b)

Fig. 2 Smoothed (100-point window), downsampled average center of mass displacement, with standard error, over 10 runs with 15 modules learning to move right, as compared to distributed GAPS (DGAPS) and DGAPS with given common mean reward (R). (a) The modules run the agreement algorithm on R only at each timestep t or to convergence at the end of episodes. (b) The modules run the agreement algorithm on both R and experience at end of episodes with original vs. scaled learning rates

on an imaginary 2D grid, viewed from above; and each module can observe at each of its 4 faces (positions 1, 3, 5, and 7 on the grid) whether or not there is another connected module on that neighboring cell. A module (call it M) can also ask those neighbors to confirm whether or not there are other connected modules at the corner positions (2, 4, 6, and 8 on the grid) with respect to M. These eight bits of observation comprise the local configuration that each module perceives. Thus, the module M in the Fig. 1c knows that lattice cells number 4–6 have other modules in them, but lattice cells 1–3 and 7–8 are free space3 . M has a repertoire of 9 actions, one for moving into each adjacent lattice cell (face or corner), and one for staying in the current position. Modules may not move into lattice positions occupied by other modules or the wall. The robot’s abstract kinematics are defined by two possible movements for each module: the sliding and convex transitions shown in Fig. 1a and b. The robot has to remain tethered by at least one module to some point on the base wall. Each module is limited in its learning ability by the partial observability of the robot configuration, as it only perceives the local window shown. The policy it is learning is represented by a table of parameters, where each entry corresponds to a possible observation-action pair, as described earlier. The learning task is locomotion along the horizontal x axis by self-reconfiguration. Each module i receives a local reward Ri = xiT − xi0 at the end of the episode (time T ), which measures its total horizontal displacement. 3

In Fig. 1c, the module M can only guess that cells 2 and 8 are free space, since no neighbors are available to provide that information. From the point of view of available legal actions, the ‘empty’ guess will always be correct.

374

P. Varshavskaya et al.

no. stuck

mean R = 2.4, agreed Rˆ = 0.63 (a)

DGAPS DGAPS with common R agreement at each t agreement at end episode

10±0.0 0.4±0.2 0.5±0.2 0.2±0.1

(b)

Fig. 3 Effect of active agreement on reward during learning: (a) The modules with greatest rewards in such configurations have least neighbors (module 13 has 1 neighbor and a reward of 5) and influence the agreed-upon value Rˆ the least. The modules with most neighbors (modules 1-10) do not have a chance to get any reward, and influence Rˆ the most. (b) Mean number of stuck configurations, with standard error, out of 10 test runs each of 10 learned policies, after 15 modules learned to locomote eastward for 100,000 episodes

4.2 Experimental Results The purpose of the experiments was to discover how agreement on common rewards and/or experience affects the speed and reliability of learning with distributed GAPS. As in earlier distributed experiments in this setting [16], the learning rate α here was initially set at the maximum value of 0.01 and steadily decreased over the first 7,000 episodes to 0.001, and remained at that value thereafter. The temperature parameter β was initially set to a minimum value of 1 and increased steadily over the first 7,000 episodes to 3, and remained at that value thereafter. This encouraged more exploration during the initial episodes of learning.

4.3 Agreement on Reward Figure 2a demonstrates the results of experiments with the learning modules agreeing exclusively on the rewards Rˆ generated during each episode. The two conditions diverged on the time at which exchanges of reward took place: at the end of the learning episode versus at each timestep t during policy execution. In the latter case, averaging stops with the end of the episode even if agreement has not yet converged to a common value, which results in lower rewards. However, we see that despite this early stopping point, not-quite-agreeing on a common reward is significantly better than using each module’s individual estimate, as predicted. Statistical analysis shows no significant difference between DGAPS with given common (mean) R and agreement run at the end of each episode. Agreement (two averaging exchanges at every timestep t) run during locomotion results in a significantly later learning convergence measure for that condition. Basic DGAPS with no agreement fails to learn anything beyond essentially a random policy. Note DGAPS with agreement at the end of each episode seems to learn better than DGAPS with given common average R for much of the learning curve. There could be a benefit to estimating Rˆ = R based on all-neighbor averaging. Consider 15

Efficient Distributed Reinforcement Learning through Agreement

15

basic DGAPS (1) centralized GAPS (2) DGAPS with agreement on rewards and experience (3)

10

10 2

smoothed average displacement

average displacement

20

3 2

5 1 0 0

2000

4000 6000 episode

8000

10000

8

375

basic DGAPS (1) centralized GAPS (2) DGAPS with agreement on reward and experience (3)

2

3

6

4

3 2 1

2

0

2000

4000 6000 episode

(a)

8000

10000

(b)

Fig. 4 Comparing centralized GAPS, DGAPS and DGAPS with agreement on experience: (a) smoothed (100-point window) average center of mass displacement over 10 trials with 15 modules learning to move right (T=50), (b) same for 20 modules (T=100): learning was done starting with α = 0.2 and decreasing it uniformly over the first 1,500 episodes to 0.02

modules in a typical non-gait configuration that will nonetheless generate a positive reward and may lead the modules into a bad local optimum in policy space (Fig. 3a). Assume the modules started in a 5x3 rectangular configuration with module 1 at the location labeled ‘start’. The average robot displacement here is R = 2.4. However, if instead of being given that value, modules actively run the synchronous agreement at the end of this episode, they will eventually arrive at the value of Rˆ = 0.63. This discrepancy is due to the fact that most of the modules, and notably those that have on average more neighbors, have received zero individual rewards. Those that have, because they started forming an arm, have less neighbors and therefore ˆ which results in smaller updates for their policy parameters, and less influence on R, ultimately with less learning “pull” exerted by the arm. Our hypothesis was therefore that, during test runs of policies learned by DGAPS with common reward versus DGAPS with agreement algorithms, we will see significantly more dysfunctional stuck configurations in the first condition. The table in figure 3b details the number of times, out of 10 test trials, that policies learned by different algorithms were stuck in such configurations. Our findings do not support our hypothesis: both types of agreement, as well as the baseline algorithm with common reward, generate similar numbers of non-gait policies.

4.4 Agreement on Reward and Experience For the set of experiments involving agreement on experience as well as rewards, the learning rate α was scaled by a factor of 10 to approximately account for the averaging of observation and observation-action pair counts4. Figure 2b demonstrates that including exchanges of experience counts results in dramatic improvement in 4

This scaling does not in itself account for the difference in experimental results. Scaling up α in the baseline DGAPS algorithm does not help in any way.

376

P. Varshavskaya et al.

Table 1 Mean number of non-gait local optima, with standard error, out of 10 test trials for 10 learned policies. The starting learning rate was α = 0.1 for 15 modules and α = 0.2 for 20 modules

Distributed GAPS (DGAPS) Centralized GAPS DGAPS with agreement on Ri and Ci

15 mods, T=50

20 mods, T=100

10±0.0 0.1±0.1 1.1 ±1.0

10±0.0 5±1.7 6±1.6

(a)

(b) Fig. 5 Screenshots at t = 5, 25 and 45, of 20 modules executing (a) a compact gait found by learning with agreement on Ri and Ci , and (b) an unfolding gait found by centralized GAPS

how early good policies are found. There is also a significant difference in learning speed between learning with the original learning rate or scaling it to account for averaging of experience. Finally, we see in figure 4a that DGAPS with agreement on both reward and experience learns comparatively just as fast and just as well as the centralized version of GAPS for 15 modules (no significant difference between centralized and agreement groups). However, when we increase the robot size to 20 modules (and episode length to 100 timesteps) in figure 4b, all three algorithms yield different reward curves, with agreement-based distributed learning getting significantly less reward than centralized GAPS. Where does this discrepancy come from? In table 1, we see that learning in a fully distributed way with agreement on reward and experience makes the modules slightly more likely to find non-gait local optima than the centralized version. However, a closer look at the policies generated by centralized learning vs. agreement reveals another difference: gaits found by agreement are more compact than those found by central GAPS. Figure 5a demonstrates in successive frames one such compact gait, as compared to the unfolding gait normally found by centralized GAPS shown in Fig. 5b. In effect, unless updates are made according to Eq. 4, active agreement results in a modified reward function. Whether or not it is preferable to the original will depend on the task at hand.

5 Discussion and Related Work We have demonstrated that agreement-style exchanges of rewards and experience can be successfully incorporated into gradient-based policy search algorithms to

Efficient Distributed Reinforcement Learning through Agreement

377

enable fully distributed learning with no extra heuristics in lattice-based SRMRs. This result suggests that the centralized algorithms and extensions that were previously presented [15, 16] are applicable to fully distributed systems. With higher numbers of modules and unfavorable initial conditions, there is a significant difference between rewards obtained by centralized learning and distributed learning with agreement, which is primarily due to the kinds of policies that are favored by the different learning algorithms. Non-pairwise agreement on rewards generates an effect where more densely connected learners have more of an influence on the final agreed-upon value. This effectively modifies the reward function, and therefore, the policy value landscape for local search. However, discussion of reward functions is beyond the scope of this paper. There is a large body of research in agreement-style algorithms for distributed robotic and sensor network systems (e.g., [8] and citations therein); we cannot do it justice in this space. Our contribution is to combine agreement with a learning algorithms for POMDPs. Our work is very closely related to the distributed optimization algorithm described by Moallemi & Van Roy [10] for networks, where the global objective function is the average of locally measurable signals. They proposed a distributed policy gradient algorithm where local rewards were pairwise averaged, with no assumptions about the policies being learned beyond continuity and continuous differentiability. In contrast, here we are interested in a special case where ideally all individual policies would be identical. This limits the class of applicable problems, but learners can take advantage of neighbors’ experience as well as rewards. In addition to prior work on distributed GAPS [12], cooperation in distributed RL has been addressed in the fully observable domain through distributed value functions [13] and coordination graphs [5, 7]. By contrast, our approach is applicable to partially observable distributed MDPs. Practical issues in sharing experience include the required communications bandwidth and message complexity. In most cases, communication is much cheaper and easier than actuation, both in resource (e.g., power) consumption, and in terms of the learning process: learning from others’ mistakes and thus reducing potentially dangerous exploration. Additionally, depending on the policy representation, significant downsizing of required communications can be achieved using standard lossless compression schemes, or a more involved distributed protocol such as consensus propagation [11].

6 Conclusion We have developed a class of algorithms incorporating stochastic gradient ascent in policy space (GAPS) and agreement algorithms, creating a framework for fully distributed implementations of this POMDP learning technique. We demonstrated its success on the learning task of locomotion by self-reconfiguration in 2D simulated SRMRs. Our results suggest that previously published centralized GAPS work may be relevant to fully distributed implementations. In the future, we may wish to develop specific caching and communication protocols for passing just the required

378

P. Varshavskaya et al.

amount of information, and therefore requiring less bandwidth, given the policy representation and current experience. Acknowledgements. The authors gratefully acknowledge the support of The Boeing Company.

References 1. Baxter, J., Bartlett, P.L.: Infinite-horizon gradient-based policy search. J. of Artificial Intelligence Res. 15, 319–350 (2001) 2. Bertsekas, D.P., Tsitsiklis, J.N.: Parallel and Distributed Computation: Numerical Methods. Athena Scientific (1997) 3. Chang, Y.-H., Ho, T., Kaelbling, L.P.: All learning is local: Multi-agent learning in global reward games. In: Advances in Neural Information Processing Systems, vol. 16 (2004) 4. Fernandez, F., Parker, L.E.: Learning in large cooperative multi-robot domains. Int. J. of Robotics and Automation 16(4), 217–226 (2001) 5. Guestrin, C., Koller, D., Parr, R.: Multiagent planning with factored MDPs. In: Advances in Neural Information Processing Systems, vol. 14 (2002) 6. Hu, J., Wellman, M.P.: Multiagent reinforcement learning: Theoretical framework and an algorithm. In: Proc. Int. Conf. on Machine Learning, pp. 242–250 (1998) 7. Kok, J.R., Vlassis, N.: Collaborative multiagent reinforcement learning by payoff propagation. J. of Machine Learning Res. 7, 1789–1828 (2006) 8. Lynch, K.M., Schwartz, I.B., Yang, P., Freeman, R.: Decentralized environmental modeling by mobile sensor networks. IEEE Trans. on Robotics 24(3), 710–724 (2008) 9. Matari´c, M.J.: Reinforcement learning in the multi-robot domain. Autonomous Robots 4(1), 73–83 (1997) 10. Moallemi, C.C., Van Roy, B.: Distributed optimization in adaptive networks. In: Advances in Neural Information Processing Systems, vol. 15 (2003) 11. Moallemi, C.C., Van Roy, B.: Consensus propagation. IEEE Trans. on Information Theory 52(11) (2006) 12. Peshkin, L.: Reinforcement Learning by Policy Search. PhD thesis, Brown University (2001) 13. Schneider, J., Wong, W.-K., Moore, A., Riedmiller, M.: Distributed value functions. In: Proc. Int. Conf. on Machine Leanring (1999) 14. Tsitsiklis, J.N., Bertsekas, D.P., Athans, M.: Distributed asynchronous deterministic and stochastic gradient optimization algorithms. IEEE Trans. on Automatic Control AC31(9), 803–812 (1986) 15. Varshavskaya, P., Kaelbling, L.P., Rus, D.: Distributed learning for modular robots. In: Proc. Int. Conf. on Robots and Systems (2004) 16. Varshavskaya, P., Kaelbling, L.P., Rus, D.: Automated design of adaptive controllers for modular robots using reinforcement learning. Int. J. of Robotics Res. 27(3–4), 505–526 (2008)

Morphology Independent Learning in Modular Robots David Johan Christensen, Mirko Bordignon, Ulrik Pagh Schultz, Danish Shaikh, and Kasper Stoy

Abstract. Hand-coding locomotion controllers for modular robots is difficult due to their polymorphic nature. Instead, we propose to use a simple and distributed reinforcement learning strategy. ATRON modules with identical controllers can be assembled in any configuration. To optimize the robot’s locomotion speed its modules independently and in parallel adjust their behavior based on a single global reward signal. In simulation, we study the learning strategy’s performance on different robot configurations. On the physical platform, we perform learning experiments with ATRON robots learning to move as fast as possible. We conclude that the learning strategy is effective and may be a practical approach to design gaits.

1 Introduction and Related Work Conventional robots are born with a flexible control system and a fixed body. That is, the behavior of a robot can be changed simply by reprogramming the robot. However, this is not the case with the robot’s morphology. Conventional robots can therefore adapt their control to the task, but must do so under the constraints of their morphology. On the contrary, the morphology of modular robots is easy to change by reassembling the modules. Hence, the design process can be transformed, so that the control is kept unchanged, while only the morphology of the robot is changed. In this paper, we let the module’s control adapt to the morphology of the robot to give the controller some level of morphology independence. Related work on configuration independent learning is limited. However, a number of papers have explored the more general problem of adaptation in modular David Johan Christensen, Mirko Bordignon, Ulrik Pagh Schultz, Danish Shaikh, and Kasper Stoy Modular Robotics Lab, The Maersk McKinney Moller Institute, University of Southern Denmark, Campusvej 55, DK-5230 Odense M, Denmark e-mail: {david,mirko,ups,danish,kaspers}@mmmi.sdu.dk

380

D.J. Christensen et al.

robots. Here, we consider related work on adaptation, such as evolution and online learning, for tasks such as locomotion. Evolution: In modular robots a classical approach to automate behavior and morphology design is to co-evolve the robot’s configuration and control [11, 4, 6]. Although appealing, one challenge with this approach is to transfer the evolved robots from simulation to physical hardware and once transferred the robot is typically no longer able to adapt. An example of adaptation by evolution in modular robots was conducted by Kamimura et al., who evolved the coupling parameters of central pattern generators for straight line locomotion of M-TRAN self-reconfigurable robots [3]. To avoid the transference problems of evolution we utilize online learning. Learning: Most related work on robot learning utilizes some degree of domain knowledge, typically about the robot morphology, when designing a learning robot controller. In our work, we want to avoid such constraints since our modular robot may be reconfigured or modules can be added or removed. Therefore, we do not know the robot’s morphology at the design time of the controller. Our approach utilizes a form of distributed reinforcement learning. A similar approach was taken by Maes and Brooks who performed distributed learning of locomotion on a 6-legged robot [5]. The learning was distributed to the legs themselves. Similarly, in the context of multi-robot systems, distributed reinforcement learning has been applied for learning various collective behaviors [8]. To the best of our knowledge, our paper is the first to apply distributed learning to fixed-topology locomotion of modular robots. Bongard et al. demonstrated learning of locomotion and adaptation to changes in the configuration of a modular robot [1]. They used a self-modeling approach, where the robot developed a model of its own configuration by performing motor actions, which could be matched with sensor information. A model of the robot configuration was evolved to match the sampled sensor data (from accelerometers) in a physical simulator. By co-evolving the model with a locomotion gait, the robot could then learn to move with different morphologies. The work presented here is similar in purpose but different in approach: Out strategy is simple, model-less and computational cheap to allow implementation on the small embedded devices that modular robots usually are. Marbach and Ijspeert has studied online optimization of locomotion on the YaMoR modular robotic system [7]. Their strategy was based on Powell’s method, which performed a localized search in the space of selected parameters of central pattern generators. Parameters were manually extracted from the modular robot by exploiting symmetries. Online optimization of 7 parameters for achieving fast movement was successfully performed on a physical robot in roughly 15 minutes [13]. As is the case in our paper, they try to realize simple, robust, fast, model-less, life-long learning on a modular robot. The main difference is that we seek to automate the controller design completely in the sense that no parameters have to be extracted from symmetric properties of the robot. Only the robot morphology must be manually assembled from modules with identical control programs. Furthermore, in our work modules have no shared parameters (except time and reward) since learning

Morphology Independent Learning in Modular Robots

381

Algorithm 1. Learning Module Controller /* * Q[A] is the discounted expected reward R of choosing Action A. * ALPHA is the smoothing factor of an exponential moving average. * 1 − EPSILON is the proportion of “greedy” action selections. * ACCELERAT E is a boolean for turning on a heuristic. * ALPHA, EPSILON and ACCELERAT E are given as parameters to controller. */ Initialize Q[A] = R, for all A evaluated in random order loop if max(Q) < R and ACCELERATE then Repeat Action A else Select Action A with max Q[A] with prob. 1 − EPSILON, otherwise random action end if Execute Action A for T seconds Receive Reward R Update Q[A]+ = ALPHA · (R − Q[A]) end loop

is completely distributed to the modules. These properties minimize the amount of communication and simplify the implementation.

2 A Strategy for Learning Actuation Patterns The ATRON modules are simple embedded devices with limited communication and computation abilities. Therefore, the learning strategy must require a minimal amount of resources and ideally be simple to implement. In this learning scenario, the robots may decide to self-reconfigure, modules may realistically break down or be reset and modules can manually be added, removed or replaced at runtime. Hence, the learning strategy must be robust and able to adapt to such events. By utilizing a simple, distributed and concurrent learning strategy such features can be naturally inherent. We let each module learn independently and in parallel based on a single shared reward signal. The learning is life-long in the sense that there is no special learning phase followed by an exploitation phase. Learning Strategy: We utilize a very simple reinforcement learning strategy, see Algorithm 1. Initially each module executes all actions, A, in random order and initializes its action value estimation, Q[A], with the rewards received. After this initialization phase, in a learning iteration, every module will perform an action and then receive a global reward for that learning iteration. Each module estimates the value of each of its actions with an exponential moving average, which suppress noise and ensures that if the value of an action changes with time so will its estimation. The algorithm can be categorized as a T D(0) with discount factor γ = 0 and with no representation of the sensor state [14]. A module can perform a fixed number

382

D.J. Christensen et al.

of actions. Each module independently selects which action to perform based on a ε -greedy selection policy, where a module selects the action with highest estimated reward with a probability of 1 − ε and a random action otherwise. Acceleration Heuristics: Performance of a module is highly coupled with the behavior of the other modules in the robot. Therefore, the best action of a module is non-stationary. It can change over time when other modules change their action. Hence, the learning speed is limited by the fact that it must rely on randomness to select a fitter but underestimated action a sufficient number of times before the reward estimation becomes accurate. To speedup the estimation of an underestimated action we tested a heuristics to accelerate the learning: If the received reward after a learning period is higher than the highest estimation of any action, the evaluated action may be underestimated and fitter than the current highest estimated action. Note that this is not always true since the fitness evaluation may be noisy. Therefore, a simple heuristic is to repeat the potentially underestimated action, to accelerate the estimation accuracy and presumably accelerate the learning, see Algorithm 1. Controller Permutations: A robot must select one action for each of its modules, therefore, the number of different controllers are #actions#modules. For example, in this paper, we use three actions and experiment with seven different robots that must learn to select a controller from amongst 27 (two-wheeler with 3 modules) to 531.441 (walker with 12 modules) different controller permutations. Therefore, for the larger robots, brute force search is not a realistic option in simulation and practically impossible on the physical system.

3 Learning with Simulated ATRON Modules 3.1 Experimental Setup Physical Simulation: Simulation experiments are performed in an open-source simulator named Unified Simulator for Self-Reconfigurable Robots (USSR) [2]. We have developed USSR as an extendable physics simulator for modular robots. Therefore, USSR includes implementations of several existing modular robots besides the ATRON. The simulator is based on Open Dynamics Engine [12] which provides simulation of collisions and rigid body dynamics. The ATRON module [10] is comprised of two hemispheres that can rotate relative to each other. On each hemisphere a module has two passive female and two actuated male connectors, see Figure 1(a). The parameters, e.g. strength, speed, weight, etc., of the simulation model and the existing hardware platform has been calibrated to ease the transfer of controllers developed in simulation to the physical modules. Through JNI or sockets, USSR is able to run the same controllers as would run on the physical platform, however, this is not utilized here. Learning to Move: In the following experiments, every module runs identical learning controllers with parameters set to ALPHA = 0.1 and 1 − EPSILON = 0.8.

Morphology Independent Learning in Modular Robots

(a) ATRON

(b) Two-wheeler

(e) Tripedal

(f) Quadrupdal

(c) Snake-4

(g) Crawler

383

(d) Bipedal

(h) Walker

Fig. 1 Seven learning ATRON robots consisting of 3 to 12 modules

In some experiments we compare with randomly moving robots, i.e. we set 1 − EPSILON = 0.0 and do not use the acceleration heuristics. An ATRON module may perform the following three actions: • HomeStop - rotates to 0 degrees and stop • RightRotate - rotate clockwise 360 degrees • LeftRotate - rotate counterclockwise 360 degrees When performing the HomeStop action, a module will always rotate to the same home position. After a learning iteration, a module should ideally be back at its home position to ensure repeatability. Therefore, a module will try to synchronize its progress to follow the rhythm of the learning iteration. If the module is too far behind, it will return directly to its home position by taking the shortest path. Unlike the physical experiments, in these simulated experiments we utilize that reward and time can be made globally avaliable in the simulator. The reward is distance traveled by the robots center of mass in the duration of a learning iteration. A learning iteration is seven seconds long, since six seconds is the minimum time (without load) to rotate 360 degrees, the extra one second is used for synchronization. Reward = Distance traveled by robot in 7 seconds

(1)

One potential limitation with this approach is that the selected action primitives may be insufficient to control all robots, for example, snakes may require oscillating motor primitives. Robot Morphologies: Since each module is running identical programs, the only difference between the different robots is in what configuration the modules are assembled. Figure 1 shows seven ATRON robots, with different morphologies, which we used for experiments. The presented approach is limited to morphologies that do not contain closed loops of modules and we generally avoid configurations that can

384

D.J. Christensen et al.

LL SL

Action of Leg 1 and Leg 3

RL RS SS LS LR SR RR RR

SR

LR

LS

SS

RS

RL

SL

LL

Action of Leg 2 and Leg 4

(b) Path through Control Space

0.025

0.025

0.020

0.020

Velocity meterssec

Velocity meterssec

(a) Quadrupedal

0.015 0.010 0.005

0.015 0.010 0.005

0.000

0.000 0

500

1000

1500

2000

2500

3000

3500

0

500

1000

Time seconds

(c) Normal Learning

1500

2000

2500

3000

3500

Time seconds

(d) Accelerated Learning

Fig. 2 Typical simulated learning examples with and without the acceleration heuristic. (a) Eight module quadrupedal crawler (four active modules). (b) Contour plot with each point indicating the velocity of a robot performing the corresponding controller (average of 10 trials per point). The arrows show the transitions of the preferred controller of the robot. (c) And (d) shows the corresponding rewards received by the robots in duration of one hour. The horizontal lines indicate the expected velocity based on the same data as the contour plot

self-collide. The reasons for this are mainly practical and we do not consider it a principal limitation. We plan to add a control layer below the learning layer to deal with these issues.

3.2 Experimental Results and Discussion Quadrupedal: In this experiment, we consider a quadrupedal consisting of 8 ATRON modules. To simplify the analysis we disable four of the modules (i.e. stops in the home position) and only allow the four legs to be active, as indicated in Figure 2(a). Also, we force the robot to start learning from a completely stopped state by initializing Q[A] to 0.1 for the HomeStop action and to 0.0 for the other actions. Note, that this will severely prolonge the convergence speed for this experiment. Our objective is to control the experiment to investigate how the proposed learning strategy behaves on a typical robot.

Morphology Independent Learning in Modular Robots

385

Fig. 3 The velocity of a quadrupedal crawler with four active modules as a function of time. Each point is the average of 10 trials. The horizontal bars indicate average convergence time and standard deviation. Note, that accelerated learning converges significantly faster (P=0.0023) for this robot

First consider the two representative learning examples given in Figure 2. The contour plot in Figure 2(b) illustrates how the robot controller transitions to gradually better robot controllers. The controller eventually converges to one of the four optimums, which corresponds to the symmetry axes of the robot (although in one case the robot has a single step fallback to another controller). The graphs in Figure 2(c) and 2(d) shows how the velocity of the robots jumps in discrete steps, that corresponds to changes in the preferred actions of modules. Figure 3 compares the convergence speed and performance of the learning with and without the acceleration heuristic. The time to converge is measured from the start of a trial until the controller transitioned to one of the four optimal solutions. In all 20 trials the robot converged, in 4 trials the robot had short fallbacks to nonoptimal controllers (as in Figure 2(c)). On average accelerated learning converged faster (19 minutes or 1146 iterations) than normal learning (32 minutes or 1898 iterations). The difference is statistically significant (P=0.0023). Note that accelerated learning on average reaches a higher velocity, but not due to the type of gaits found. Rather the faster velocity is due to the acceleration heuristics, which tends to repeat good performing actions at the cost of random exploration. This can also be seen by comparing Figure 2(c) with 2(d). As summarized in Table 1 the learning strategy behaves in roughly the same way independent of the acceleration heuristic. A typical learning trial consists of 4-5 controller transitions, where a module changes its preferred action before the controller converges. In about 90% of these transitions it will only change the action of one module. This indicates that at a global level the robot is performing a localized random search in the controller space. Although, the individual modules are not collectively searching in any explicit manner, this global strategy emerges from the local strategy of the individual modules. Different Morphologies: An important requirement of the proposed online learning strategy is the ability to learn to move with many different robot morphologies without changing the control. In this experiment, we perform online learning with seven different simulated ATRON robots, see Figure 1. In each learning trial, the robot had 60 minutes to optimize its velocity. For each robot type 10 independent trials were performed. Results are shown in Figure 4.

386

D.J. Christensen et al.

Table 1 Average number of controller transitions to reach optimal solution, with standard deviations in parentheses. To measure the number of controller transitions very brief transitions of one or two learning steps (7-14 seconds) are censored away. The results are based on 10 trials of quadrupedal crawler with 4 active modules learning to move. Note, that there is no significant difference in the type of controller transitions. Also, 1-step transitions are by far the most common, which indicate that the search is localized

Transitions per Trial 1-Step Transitions 2-Step Transitions 3-Step Transitions 4-Step Transitions

Normal 4.4 (1.17) 87% 13% 0% 0%

Accelerated 4.0 (0.94) 90% 6% 4% 0%

Fig. 4 Velocity at the end of learning in simulation. Each bar is the average velocity (reward) from the 50 to the 60 minute of 10 independent trials. Error bars indicate one standard deviation of average robot velocity. Note that both normal and accelerated learning has an average higher velocity than random movement

Compared to randomly behaving robots, both normal and accelerated learning improves the average velocity significantly. We observe that each robot always tends to learn the same, i.e., symmetrically equivalent gaits. There is no difference in which types of gaits the normal and accelerated learning strategy finds. Overall, the learning of locomotion is effective and the controllers are in most cases identical to those we would design by hand using the same action primitives. A notable exception is the snake robot which has no good controller given the current set of action primitives. The other robots converged within 60 minutes to best-known gaits in 96% of the trials (115 of 120 trials). Convergence time was on average less than 15 minutes for those robots, although single trials would be caught in suboptimal solutions for extended time periods. We found no general trend in the how the

Morphology Independent Learning in Modular Robots

387

Fig. 5 Experimental setup of online learning

morphology affects the learned gaits. For example, there is no trend that smaller robots or larger robots are faster, except that wheeled locomotion is faster than legged locomotion.

4 Learning with Physical ATRON Robots In the previous section, we studied the configuration independent learning strategy purely in simulation. In this section, to validate our results we perform online learning on physical ATRON robots.

4.1 Experimental Setup The ATRON modules are not equipped with a sensor that allows them to measure their own velocity or distance traveled, as required for the reward signal. To compensate for this we construct a setup, which consists of an arena with an overhead camera connected to a server. Figure 5 illustrates the experimental setup. The server tracks the robot and sends a reward signal to the robot. The original ATRON module does not have wireless communication. For this (and other) reasons, we are developing a number of modified ATRON modules, which have an integrated Sun SPOT [9] and make use of its wireless communication interface. In each learning robot, a single Sun SPOT enabled ATRON module is used, which receives reward updates from the server. The Sun SPOT enabled ATRONs are in development and can currently not be actuated for reliability reasons. Instead, we place the Sun SPOT modules so that its effect on the learning results can be disregarded. The learning algorithm, as specified in Algorithm 1, is running on the modules. Each module runs identical programs and is learning independently and in parallel with other modules. With 10 Hz every module sends a message containing its current state, timestep and reward to all of its neighbors through its infrared communication channels. The timestep is incremented and the reward is updated from the server side every 7 seconds. When a new update is received, a module performs a learning update and start a new learning iteration. The state can from the server

388

D.J. Christensen et al.

Table 2 Results of online learning on two-wheeler and quadrupedal robots

Exp. 1 2 3 4 5 Total Phy. mean Sim. mean

Twowheeler Conv. Time Exp. Time (seconds) (seconds) 28 629 35 707 567 967 28 695 798 1020 1456 4018 291 804 225 —

Quadrupedal Conv. Time Exp. Time (seconds) (seconds) 1680 2401 1232 2157 448 1891 700 2236 336 2468 4396 11153 879 2231 701 —

side be set to paused or learning. The robot is paused by the server when it moves beyond the borders of the arena and is then manually moved back onto the arena before the learning is continued. In the presented results, the paused time intervals have been removed.

4.2 Experimental Results and Discussion In these experiments, learning is performed directly on the modules and only the reward signal is computed externally. We perform experiments with two different robots, a three-module two-wheeler and an eight-module quadrupedal, which has a passive ninth module for wireless communication. For each robot, we report on five experimental trials, two extra experiments (one for each robot) were excluded due to mechanical failures during the experiments. An experimental trial ran until the robot had convincingly converged to a near optimal solution. Since not all physical experiments are of equal duration, we extrapolate some experiments with the average velocity of its last 10 learning iterations to generate the graphs of Figure 6(a) and 6(c). In total, we report on more than 4 hours of physical experimental time. Two-wheeler: Table 2 shows some details for five experimental trials with a twowheeler robot. The time to converge to driving either forward or backward is given. For comparison the equivalent convergence time measured in simulation experiments is also given. In three of the five experiments, the robot converges to the best-known solution within the first minute. As was also observed in simulation trials, in the other two trials the robot was stuck for an extended period in a suboptimal behavior before it finally converged. We observe that the physical robot on average converges a minute slower than the simulated robot, but there is no significant difference (P=0.36) between simulation and physical experiments in terms of mean convergence time. Figure 6 shows the average velocity (reward given to the robot) as a function of time for the two-wheeler in both simulation and on the physical robot. The results are similar, except that the physical robot moves faster than in simulation (due to simulator inaccuracies).

Morphology Independent Learning in Modular Robots 0.05

389

0.04

Velocity meterssec

Velocity meterssec

0.04

0.03

0.02

0.03

0.02

0.01

0.01

0.00

0.00 0

200

400

600

0

800

200

400

600

800

Time seconds

Time seconds

(a) Physical Two-Wheeler

(b) Simulated Two-Wheeler

0.05

Velocity meterssec

Velocity meterssec

0.022 0.04

0.03

0.02

0.020 0.018 0.016 0.014 0.012 0.010

0.01 0

500

1000

1500

Time seconds

(c) Physical Quadrupedal

2000

0

500

1000

1500

2000

Time seconds

(d) Simulated Quadrupedal

Fig. 6 Average velocity of five trials as a function of time for both physical and simulated experiments for a two-wheeler and a quadrupedal. Points are the average reward in a given timestep and the lines indicate the trend

Fig. 7 Pictures from learning experiment with quadrupedal walker. A 7 seconds period is shown. The robot starts in its home position, performs a locomotion period, and then returns to its home position. In each of the five experiments, the quadrupedal converged to symmetrically equivalent gaits. All five gaits were equivalent to the gaits found in simulation

Quadrupedal: Pictures from an experimental trial is shown in Figure 7, where a 9-module quadrupedal (8 active modules and 1 for wireless communication) learns to move. Table 2 summarized the result of five experimental trials. In all five trials, the robot converges to a known best gait. The average convergence time is less than 15 minutes, which is slower than the average of 12 minutes it takes to converge in simulation. The difference is, however, not statistical significant (P=0.29). Figure 6 shows the average velocity versus time for both simulated and physical experiments with the quadrupedal. We observe that the measured velocity in the physical trials contains more noise than the simulated trials. Further, the physical robot also achieves a higher velocity than in simulation (due to simulator inaccuracies).

390

D.J. Christensen et al.

Another observation we made was that the velocity difference between the fastest and the second fastest gait is smaller in the real experiments than in simulation, which together with the extra noise may explain why the physical trial on average converges almost 3 minutes slower than in simulation.

5 Extensions and Future Work In addition to the experiments presented here, we already did simulated experiments on the strategy’s scalability, tolerance of module failures, adaptation after self-reconfiguration and its application to other types of modular robots. These experiments are left out due to limited space. Here, we will just mention that based on these experiments: i) The strategy scaled up to a 60 module robot, with learning divergence becoming increasingly significant. ii) The strategy seamlessly adapted to failed modules or a new morphology after self-reconfiguration. iii) The strategy was extended to learn gait control tables to enable learning of M-TRAN robots (and thereby most modular robots). Future work will present and extend these results.

6 Conclusion In this paper, we explored an online learning strategy for modular robots. The learning strategy is simple to implement since it is distributed and model-less. Further, the strategy allows us to assemble learning robots from modules without changing any part of the program or putting severe constraints on the types of robot morphologies. In simulation we studied a learning quadrupedal crawler and found that from its independently learning modules, a higher-level learning strategy emerged, which was similar to localized random search. We performed experiments in simulation of ATRON modules, which indicate that the strategy is sufficient to learn quite efficient locomotion gaits for a large range of different morphologies up to 12-module robots. A typical learning trial converged in less than 15 minutes depending on the size and type of the robot. Further, we performed experiments with physical ATRON robots online learning to move. These experiments validated our simulation results. In conclusion, the proposed learning strategy may be a practical approach to design locomotion gaits.

References 1. Bongard, J., Zykov, V., Lipson, H.: Resilient machines through continuous selfmodeling. Science 314(5802), 1118–1121 (2006) 2. Christensen, D.J., Schultz, U.P., Brandt, D., Stoy, K.: A unified simulator for selfreconfigurable robots. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (2008)

Morphology Independent Learning in Modular Robots

391

3. Kamimura, A., Kurokawa, H., Yoshida, E., Murata, S., Tomita, K., Kokaji, S.: Automatic locomotion design and experiments for a modular robotic system. IEEE/ASME Transactions on Mechatronics 10(3), 314–325 (2005) 4. Lipson, H., Pollack, J.B.: Automatic design and manufacture of robotic lifeforms. Nature 406, 974–978 (2000) 5. Maes, P., Brooks, R.A.: Learning to coordinate behaviors. In: National Conference on Artificial Intelligence, pp. 796–802 (1990) 6. Marbach, D., Ijspeert, A.J.: Co-evolution of configuration and control for homogenous modular robots. In: Proc. 8th Int. Conf. on Intelligent Autonomous Systems, Amsterdam, Holland, pp. 712–719 (2004) 7. Marbach, D., Ijspeert, A.J.: Online Optimization of Modular Robot Locomotion. In: Proceedings of the IEEE Int. Conference on Mechatronics and Automation (ICMA 2005), pp. 248–253 (2005) 8. Mataric, M.J.: Reinforcement learning in the multi-robot domain. Auton. Robots 4(1), 73–83 (1997) 9. Sun Microsystems. Sun spot project, http://www.sunspotworld.com/ 10. Østergaard, E.H., Kassow, K., Beck, R., Lund, H.H.: Design of the atron lattice-based self-reconfigurable robot. Auton. Robots 21(2), 165–183 (2006) 11. Sims, K.: Evolving 3d morphology and behavior by competition. In: Brooks, R., Maes, P. (eds.) Proc. Artificial Life IV, pp. 28–39. MIT Press, Cambridge (1994) 12. Smith, R.: Open dynamics engine (2005), www.ode.org 13. Sproewitz, A., Moeckel, R., Maye, J., Ijspeert, A.: Learning to move in modular robots using central pattern generators and online optimization. Int. J. Rob. Res. 27(3-4), 423– 443 (2008) 14. Sutton, R.S., Barto, A.G.: Reinforcement Learning - An Introduction. MIT Press, Cambridge (1998)

Reconfigurable Modular Robots Adaptively Transforming a Mechanical Structure (Numerical Expression of Transformation Criteria of “CHOBIE II” and Motion Experiments) Y. Suzuki, N. Inou, M. Koseki, and H. Kimura

1

Abstract. This study describes a group robotic system “CHOBIE II.” The feature of CHOBIE II is a function to form a self-reconfigurable structure. We focus on a motion planning method to obtain control algorithms. For this purpose, we introduce a numerical criterion for generating transformations. The criterion is expressed with a matrix form of 32 parameters. The criteria facilitate implementation of various motions of CHOBIE II, not requiring a high information processing unit. We demonstrate two types of transformations of CHOBIE II with the numerical expression. It is confirmed that the proposed method has advantages of simplification and downsizing of control algorithms of the modules. Keywords: Modular robots, Self-reconfiguration, Distributed autonomous systems.

1 Introduction A self-reconfigurable modular robotic system is composed of coalition of multiple robotic units having their own CPUs and actuators. Such a system can carry out a variety of tasks by changing the shape or function of itself [1, 2, 3, 4, 5, 6]. Furthermore, if a control program is set so that the system changes them according to its circumstances, then it can work adaptively in alien environments or time-varying environments. To realize a practical modular robotic system, we must examine control algorithms which enable cooperative motions as well as mechanical designs of them. In addition, Y. Suzuki, N. Inou, M. Koseki, and H. Kimura Department of Mechanical and Control Engineering, Tokyo Tech, 2-12-1, O-okayama, Meguro-ku, 152-8552 Japan e-mail: [email protected]

394

Y. Suzuki et al.

for adaptability of the system to the circumstances, it requires a scheme that all the modules detect their surroundings and switches their motion patterns flexibly. There are some modular robots having adaptability to terrains by changing their configurations or motion patterns, such as Polybot [5], M-TRAN [3], ATRON [4], etc. Correspondingly, we have developed modular robots called “CHOBIE II” forming a mechanical structure which has adaptability to load conditions [7, 8]. CHOBIE II has following properties. • CHOBIE II consists of identical modules having a block-like shape. • The modules construct a 2-D structure as shown in Fig. 1. • There is no host device like an external computer or an especially high-graded module. • The structural transformation is performed by cooperative slide movements of the modules. • Adjacent modules communicate each other by ON/OFF signals of infra-red LEDs. • Each module determines its own actions by the internal microcomputer. • The same control program is installed in all of them. Construction of a 2D lattice structure

The modules of CHOBIE II

Fig. 1 Modular robot “CHOBIE II” constructing a self-reconfigurable structure

In the previous paper, we introduced a new control method for group robots, “temporary leader scheme,” inspired by a behavior of swarms of fishes or birds [6]. It is confirmed by simulations and actual experiments that the modules can perform cooperative structural transformations by control programs based on the scheme. In this paper, we propose a method to express the criteria with a matrix form. Then, the criteria become to be simply expressed, so it is expected that the control programs can be more easily developed. Additionally, there is also a merit that the method reduces the data size of the control program installed in the ROM of the microcomputers. As a result, the method improves practicality of CHOBIE II.

2 Mechanical Structure of CHOBIE II In the system of CHOBIE II, a 2-D lattice structure is constructed by the block-like shaped modules. For the purpose of construction of a strong structure, each module has a mechanism with low degree of freedom, and a specialized algorithm is applied for control of the modules. In this chapter, we describe mechanisms for connection and transformation of the modules, and then we explain the control algorithm to carry out the structural transformations.

Reconfigurable Modular Robots Adaptively Transforming a Mechanical Structure

395

2.1 Mechanism of the Module Figure 2(a) shows the mechanism of the CHOBIE II module. Each module has convex parts on upper and right sides, and concave parts on lower and left sides. Neighboring modules connect with the associated parts each other as shown in Fig. 2(b), and a lattice structure is constructed in the 2-D plane. There are two sets of slide motion mechanisms allocated in vertical and horizontal directions, which are driven by DC motors and generate driving forces in the two directions. Due to these mechanisms, the modules construct a structure and transform the structure by the slide movements. All of control devices such as motor drivers, a microcomputer and batteries are embedded inside of the module.

Fig. 2 Slide motion mechanism (a), connection of two modules (b) and structural transformation (c) of CHOBIE II (There is a mechanical constraint in connection of adjacent modules)

Additionally, there are communication devices composed of infra-red LEDs and photo-transistors. One LED and one photo-transistor are set on each of four contact surfaces, and they are allocated so that an LED of a module faces to a photo-transistor of another module when the two modules are neighboring. By turning ON/OFF the LEDs, the modules send signals to neighboring modules. The specifications of the module are shown in table 1. A feature of the design of CHOBIE II is that the constructed structure is rigid, and the transformations can be carried out even if an external load acts on the structure. On the other hand, because of restriction among the modules, there is the Table 1 Specifications of the Module Main material

ABS resin

Size

96 x 96 x 100 [mm]

Mass

0.59 [kg]

Driving force

20 [N]

Moving velocity

Upward…44.4 [mm/s] Downward…57.1 [mm/s] Sideways…53.3 [mm/s]

MPU

H8/3664F (Clock frequency: 16 [MHz], ROM: 32 [KB])

Battery

Li-355SP x 2 (serial) (7.4 [V], 550 [mAh])

396

Y. Suzuki et al.

mechanical constraint that connection and disconnection of modules have to be performed by the slide movements along the contact surfaces (ref. Fig. 2(c)). Thus, during a transformation of the structure, only modules on a specific “row” or “column” must move in the same direction at the same time. For such a cooperative motion, a control algorithm considering the mechanical constraint is required.

2.2 Control Algorithm In order to satisfy the mechanical constraint of CHOBIE II, we have proposed “temporary leader scheme” as a control method for distributed systems. In this method, the same communication rules and information of specified patterns of local shapes are given to the modules, and they generate transformations by following steps. 1. All the modules communicate and obtain positional information around themselves according to the communication rules. 2. A module becomes a leader if it is in a particular condition of local shapes. (The other modules switch to a mode to wait for a signal of drive command.) 3. The leader sends a signal of drive command to generate a transformation corresponding to the condition of the local shape. 4. Modules which received the signal drive their motors synchronously and carry out the slide movement. 5. After the transformation, the next leader is selected with the same rules. By repeating these steps, the modules generate successive transformations. In this method, any module equally has possibility to become a temporary leader, and a module which satisfies a specific condition in each situation becomes a leader.

3 Numerical Expression of Criteria for Generating Transformations In this paper, we introduce a numerical expression of criteria for generating transformations. Here, a criterion regulates what kinds of transformations should be carried out in each condition. In this method, the criterion is expressed by a set of parameters, and we can numerically describe induced tendencies of generating transformations of the structure. Therefore, it becomes easy to develop control algorithms.

3.1 Expression by 32 Parameters As a fundamental concept, CHOBIE II generates a transformation when an undesirable local characteristic exists in the structure, and the transformation is generated so as to remove the characteristic. Therefore, it is important to define what kinds of characteristics are undesirable. In this method, each module focuses on characteristic local shapes on its row and column, and evaluated values of “undesirability” of these shapes regulate a criterion for generating transformations.

Reconfigurable Modular Robots Adaptively Transforming a Mechanical Structure

397

First, each module gathers information about states of both ends of its row and column, and existences of characteristic local shapes are described in two kinds of four dimensional vectors (cv: vertical characteristic vector, ch: horizontal characteristic vector). cv and ch are determined as shown in Fig. 3. Here, following 8 kinds of characteristic shapes are focused on.

Fig. 3 Example case of determination of cv and ch

• • • • • • • •

Upper end of the column is left end (cv1) Upper end of the column is right end (cv2) Lower end of the column is left end (cv3) Lower end of the column is right end (cv4) Left end of the row is upper end (ch1) Left end of the row is lower end (ch2) Right end of the row is upper end (ch3) Right end of the row is lower end (ch4)

Second, about each of the 8 shapes, undesirability of combined states with other 4 shapes is numerically estimated. That is, undesirability of 32 kinds of states is given in numerical values. Each module calculates undesirability of the state on its row and column with the following equation, t

t

L = cv Avch + cv Ah ch = {cv1 cv 2

+ {cv1 cv 2

cv 3

cv 3

⎡α v11 ⎢α cv 4 }⎢ v 21 ⎢α v31 ⎢ ⎣α v 41 ⎡α h11 ⎢α cv 4 }⎢ h 21 ⎢α h 31 ⎢ ⎣α h 41

α v12 α v 22 α v32 α v 42

α v13 α v 23 α v 33 α v 43

α h12 α h 22 α h 32 α h 42

α h13 α h 23 α h33 α h 43

α v14 ⎤ ⎧ ch1 ⎫ α v 24 ⎥⎥ ⎪⎪ch 2 ⎪⎪ ⎨ ⎬ α v 34 ⎥ ⎪ch 3 ⎪ ⎥⎪ ⎪ α v 44 ⎦ ⎩ch 4 ⎭

α h14 ⎤ ⎧ ch1 ⎫ α h 24 ⎥⎥ ⎪⎪ch 2 ⎪⎪ ⎨ ⎬ α h 34 ⎥ ⎪ch3 ⎪ ⎥⎪ ⎪ α h 44 ⎦ ⎩ch 4 ⎭

(3.1)

where, “Αv” and “Αh” are 4 x 4 matrices which express the undesirability of the 32 states. The elements of these matrices are corresponding to the 32 kinds of states in Fig. 4. For example, the (1, 1) elements of the both matrices give the undesirability of the condition that “upper end of its column is left end” and “left end of its row is

398

Y. Suzuki et al.

Local shapes about upper end of the structure

Local shapes about left end of the structure

Local shapes about lower end of the structure

Local shapes about right end of the structure

Fig. 4 Combinations of characteristics indicated by the parameters ((a) is corresponding to Av, and (b) is corresponding to Ah)

upper end.” And “L,” which is calculated by them, gives the undesirability of each position of modules. We call it “leader index.” Third, a module which has the largest value of leader index becomes a leader. Then, the leader generates the transformation to dissolve the most undesirable characteristic state around itself, which mostly contributed in the determination of the leader. By use of the method described above, various motions of CHOBIE II can be performed by setting the 32 parameters. In addition, this method realizes scalability of the system because each module is concerned with states of only four modules at the both ends of its row and column. That is, amount of information processed by a module is independent of the total number of the modules in the structure. Although time required for collecting the information is dependent of the scale, it is just a short time. Thus, it does not limit the scalability of the system in practical use.

3.2 Coding of Control Program We code a control program for CHOBIE II applying the numerical expression of criteria. First, each module has to gather information about both ends of its row and column to derive cv and ch. It is obtained as follows (ref. Fig. 5). 1. All the modules turn ON the LEDs of upper, lower, left and right side. ⇒ The modules confirm from the reception condition whether they are upper end, lower end, left end or right end. ⇒ All the modules turn OFF the LEDs. 2. Modules on upper ends turn ON the LED of lower side if they are on left ends. The other modules turn ON the LEDs of lower side if they receive signals from upper sides. ⇒ In this time, if a module receives a signal from upper side, it makes cv1 = 1, otherwise it makes cv1 = 0. ⇒ All the modules turn OFF the LEDs. 3. Modules on upper ends turn ON the LED of lower side if they are on right ends. The other modules turn ON the LEDs of lower side if they receive signals from upper sides.

Reconfigurable Modular Robots Adaptively Transforming a Mechanical Structure

399

⇒

In this time, if a module receives a signal from upper side, it makes cv2 = 1, otherwise it makes cv2 = 0. ⇒ All the modules turn OFF the LEDs. 4-9. Hereinafter, by analogous information processing from lower ends, left ends and right ends, the modules gather information of local shapes. 2)

Upper ends of these columns are left ends.

3)

Upper ends of these columns are left ends.

Cv1 = 1

4)

Lower ends of these columns are left ends.

Cv2 = 1

5)

Lower ends of these columns are left ends.

Cv3 = 1

6)

Left ends of these rows are upper ends.

Cv4 = 1

7)

Left ends of these rows are lower ends.

Ch1 = 1

8)

Right ends of these rows are upper ends.

Ch2 = 1

9)

Right ends of these rows are lower ends.

Ch3 = 1

Ch4 = 1

Fig. 5 Information processing about characteristic configurations

Using cv, ch obtained as above, and Av, Ah defined in advance, each module calculates its leader index L by Eq. (3.1). Next, the modules search for the module which has the largest value of leader index. Since the possible range of L is regulated by the parameters of Av and Ah, the modules can specify the largest value of L by a method similar to the bisection method. Then, a module having the largest value of L becomes a leader. By the procedure, the modules can determine a leader appropriately even by local communications.

4 Motion Experiments In this chapter, we demonstrate some motions of CHOBIE II performed by setting the 32 parameters appropriately.

4.1 Crawl Motion The first motion is the locomotion on a flat ground called “Crawl motion,” shown in Fig. 6. Crawl motion is carried out by 4 patterns of transformation modes shown in Fig. 7. Each of the modes can be induced by one element of matrix parameters. Therefore, we set the parameters as Eq. (4.1), where the 4 patterns of transformation modes are generated by the 4 parameters set in “1” in the matrices.

400

Y. Suzuki et al.

Fig. 6 Crawl motion

⎡1 ⎢0 Av = ⎢ ⎢0 ⎢ ⎣0

0 0 0⎤ ⎡0 0 ⎢0 0 0 0 0⎥⎥ , Ah = ⎢ ⎢0 1 0 0 0⎥ ⎥ ⎢ 0 0 1⎦ ⎣0 0

0 0⎤ 1 0⎥⎥ 0 0⎥ ⎥ 0 0⎦

(4.1)

Fig. 7 Transformation modes for crawl motion

We installed a control program with these parameters to the microcomputers of the modules, and demonstrated the motion experiment of structural transformations using actual modules. As shown in Fig. 8, the modules successfully performed crawl motion. 1

2

3

4

6

7

8

9

5

Fig. 8 Crawl motion operated by 5 modules

4.2 Stair-Like Transformation We also tried the motion to transform the structure from the initial configuration to the goal configuration shown in Fig. 9. Initial Configuration

Fig. 9 Objective structural transformation

Goal Configuration

Reconfigurable Modular Robots Adaptively Transforming a Mechanical Structure

401

This motion is realized by 3 patterns of transformation modes shown in Fig. 10; the mode (a) generates a transformation to move the left end of the structure upward, the mode (b) is to move the upper end of the structure left, and the mode (c) is to move the left end of the structure downward. Therefore, we set the parameters as Eq. (4.3). The result of the experiment is shown in Fig. 11, where the objective configuration of the structure is successfully constructed. ⎡0 ⎢0 Av = ⎢ ⎢0 ⎢ ⎣0

0 0 0⎤ ⎡0 ⎢0 0 0 0⎥⎥ , Ah = ⎢ ⎢0 1 0 0⎥ ⎥ ⎢ 0 0 0⎦ ⎣0

0 0 0⎤ 0 4 0⎥⎥ . 0 2 0⎥ ⎥ 0 0 0⎦

(4.3)

Fig. 10 Transformation modes for constructing the goal configuration

1

2

3

4

6

7

8

9

5

Fig. 11 Experimental result with the control program using parameter in Eq. (4.3)

The control programs are written into the ROM of the microcomputers of the modules. Although the size of the ROM is limited, in this method, one motion pattern is expressed by only 32 parameters, so we can install various motion patterns into it. We discuss this merit in the next chapter.

5 Discussion By use of the proposed method, we can give a criterion for structural transformations of CHOBIE II by only setting 32 parameters. There are following two advantages. 1. Since a criterion to perform one motion is expressed by a little amount of information, multiple patterns of motions can be installed into the modules. 2. Since a criterion is expressed in a simple manner (by 32 numerical values), it is easy to derive an appropriate criterion to perform an objective motion. We now use H8/3664F with 32KB of ROM as the microcomputer of the module. In the previous method, a control algorithm takes 8 – 12KB of memory to perform one motion pattern. Therefore, only 2 or 3 patterns of motion can be installed in the

402

Y. Suzuki et al.

microcomputer. On the other hand, in the new method, although it requires about 20KB for basic communication process, each criterion is described in a very few memory. It is easy to install dozens of motion patterns to the module. Therefore, if the modules equip sensors, and if the control program is described so as to switch their motion pattern according to the sensor inputs, CHOBIE II is able to change its behavior autonomously adapting to its circumstance. For example, it is expected that, if the modules have load sensors and perceive the mechanical states of the structure, the modules can effectively reinforce the weak points of the structure. And, about the advantage (2), the new method focuses on undesirability of each local shape rather than the transformation process of the whole structure. Thus, only by setting what kinds of local shapes are undesirable, we can realize various objective motions. For example, a transformation to an objective configuration is operated by setting the parameters so that local shapes which do not exist in the objective configuration are undesirable. Here, the modules continue structural transformations until they reach to the objective configuration (or a configuration which has the same features as the objective configuration). Based on the idea, we are now under consideration of a numerical solution to derive parameters for generating arbitrary objective motions. Additionally, in comparison to the previous method which regulates a transformation process, the new method of numerical expression of a criterion is more robust.

6 Conclusion In this study, we introduced a method to numerically express a criterion of modular robotic system, CHOBIE II, which constructs a self-reconfigurable structure. The criterion is expressed by 32 parameters corresponding to undesirability of local shapes on row and column of each module. We confirmed that CHOBIE II can actually perform various motions if the parameters are appropriately set by simulation and experimental results. The method has potential to generate various motion patterns with 32 parameters. Therefore, by inputting many sets of parameters for motion patterns into the modules, and by setting a control program so as to use them as the situation demands, CHOBIE II will have high adaptability to various mechanical environments.

References 1. Akiya, K., Haruhisa, K., Eiichi, Y., Satoshi, M., Kohji, T., Shigeru, K.: Automatic Locomotion Design and Experiments for a Modular Robotic System. IEEE/ASME Transactions on Mechatronics 10(3), 314–325 (2005) 2. David, B.: Comparison of A* and RRT-Connect Motion Planning Techniques for Self-Reconfigurable Planning. In: Proc. of the 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2006), pp. 892–897 (2006)

Reconfigurable Modular Robots Adaptively Transforming a Mechanical Structure

403

3. Eiichi, Y., Haruhisa, K., Akiya, K., Satoshi, M., Kohji, T., Shigeru, K.: Planning Behaviors of Modular Robots with Coherent Structure using Randomized Method. In: Proc. of the 7th International Symposium on Distributed Autonomous Robotic Systems (DARS 2004), pp. 2056–2061 (2004) 4. Morten, W.J., Esben, H.O., Henrik, H.L.: Modular ATRON: Modules for a self-reconfigurable robot. In: Proc. of the 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2004), pp. 2068–2073 (2004) 5. Yim, M., Zhang, Y., Duff, D.: Modular Robots. IEEE Spectrum (2002) 6. Daniela, R., Zack, B., Keith, K., Marsette, V.: Self-reconfiguring Robots. Communications of the ACM 45(3) (2002) 7. Michihiko, K., Kengo, M., Norio, N.: Cellular Robots Forming a Mechanical Structure – Evaluation of structural formation and hardware design of “CHOBIE II”. In: Proc. Of 7th International Symposium on Distributed Autonomous Robotic Systems (DARS 2004), pp. 131–140 (2004) 8. Yosuke, S., Norio, I., Hitoshi, K., Michihiko, K.: Reconfigurable group robots adaptively transforming a mechanical structure – Crawl motion and adaptive transformation with new algorithms. In: Proc. of the 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2006), pp. 2200–2205 (2006)

Toward Flexible and Scalable Self-reconfiguration of M-TRAN Haruhisa Kurokawa, Kohji Tomita, Akiya Kamimura, and Satoshi Murata

Abstract. Scalable self-reconfiguration of an M-TRAN modular robotic system is presented herein. Possible configurations are restricted to those of a symmetric planar type, which are made by an arbitrary number of M-TRAN modules. A logical model of such configurations is made in which each module is represented as a square cell, where self-reconfiguration is represented by cooperative motions of two neighbor cells. Based on the concept, programs for simulations and experiments were developed. Experiments of simple selfreconfigurations using local communication between modules by infrared light were successful.

1 Introduction A self-reconfigurable modular robot, as a distributed autonomous system, is expected to be flexible and robust. Even with modules having limited ability of motion, control, sensing and processing, cooperation of huge quantities of modules enables flexible, diverse, and adaptive behaviors as a whole. Moreover, system scalability is advantageous for robustness and fault-tolerance: the number of modules is variable in real time and total functionality is flexible with regard to the number. These features, however, have been mere objectives of simulation studies, they are yet unrealized by hardware and experiments. Various problems have Haruhisa Kurokawa, Kohji Tomita, and Akiya Kamimura National Institute of Advanced Industrial Science and Technology (AIST), Tsukuba, Ibaraki, 305-8564 Japan e-mail: {kurokawa-h,k.tomita,kamimura.a}@aist.go.jp Satoshi Murata Interdisciplinary Graduate School of Science and Engineering, Tokyo Institute of Technology, Midori-ku, Yokohama 226-8502 Japan e-mail: [email protected]

406

H. Kurokawa et al.

arisen in module design, hardware realization, and control. Although several three-dimensional systems [1, 2, 3, 4] and various algorithms [5, 6, 7] have been studied, gaps between software and hardware studies are very large. A typical cubic module used in most software studies is rather unsuitable to hardware realization because it tends to be mechanically complex and heavy. Successful hardware, such as that of Modular Transformer (M-TRAN) [2] and ATRON [3], is based on symmetry reduction; such designs have a drawback in their limited flexibility of self-reconfiguration. The M-TRAN is a modular system that can actually self-reconfigure. Various three dimensional self-reconfigurations were achieved by its third prototype and using various algorithms [5]. Most self-reconfigurations in the experiments, however, were deterministic and not scalable. For that reason, they were neither autonomous nor robust. This paper describes our trial toward scalable and autonomous selfreconfiguration. A type of regular planar structure comprising an arbitrary number of modules is modeled as square cells and distributed control of a self-reconfiguration is represented by state transitions of cells. We have been developing multilayer programs, the upper layers of which are compatible for simulation and control. In both simulations and experiments, only local communications between neighbor modules were used for the sake of the system’s scalability. Various other studies of algorithms and simulations addressed similar square lattice systems, but the cell model for M-TRAN is not identical to the ideal models for such studies. For that reason, the developed algorithms and programs were rather complex. Such differences, along with hardware problems identified by experiments, are worth consideration in future software studies of self-reconfigurable robots. In the following section, the basic features of the M-TRAN module are described briefly. The research objectives and goals of this study are defined. A cell model and decision-making are detailed in Sections 3 and 4, and programs for simulation and control are described in Section 5. In the final two sections, experimental results and future studies are presented with concluding remarks.

2 M-TRAN and Self-reconfiguration Task 2.1 Design and Hardware The M-TRAN system is based on a cubic lattice, with each module composed of a pair of semi-cubes (Fig. 1). This design is less symmetric than an ideal cubic module, but it requires a simpler mechanism. An M-TRAN module has two rotational joints of 180-degree range, and can connect with other modules by six flat surfaces for connection. Each connection surface has electric contacts for a Controller Area Network (CAN) bus, and transmitters and receivers of infrared (IR) light used for proximity sensing and communication. In a lattice structure made of M-TRAN modules, each module’s posture is

Toward Flexible and Scalable Self-reconfiguration of M-TRAN Active (male) block

f2

Passive (female) block

Link

Photo-diode

407

f1

Photo-detector

P0

f0 Photo-detector

m0

m2 (m0)

f0 (f2)

P1

Photo-diode m1

P0

m2

f0

m1

P2

P2

Contact pin

P1

180 Rotation m2

Connection surface : m0, m1, m2, f0, f1, f2

(a) Module hardware

P3

(b) Posture

P1

(c) Posture change

Fig. 1 M-TRAN module. (a) Hardware. Each semi-cube fits in a 65 mm cube and a module weighs 420 g. (b) Nominal posture with discrete joint angles for lattice structures. (c) Sequence of step motions for collision avoidance

one of four postures shown in Fig. 1(b) considering symmetric equivalence. Motions of lattice type self-reconfiguration are described using a series of changes between these postures with 90 degree step rotations and connection changes. Therefore, they are represented as a series of discrete state changes. For some changes between two postures such as the top and the bottom in Fig. 1(c), transient motions are necessary for collision avoidance. Various self-reconfigurations have been produced using up to 24 modules, but hardware problems persist, such as alignment error, momentary breakage of bus lines, and computer resetting by transient power voltage drops [2]. The latter two are attributable merely to the current design, but the first is common to most three-dimensional modular systems. The M-TRAN modules mutually communicate either via CAN bus or using IR light. Fast global bus communication such as CAN is usually necessary for time-critical tasks such as dynamic walking. However, it is unnecessary for quasi-static motions and self-reconfiguration. Moreover, a global bus is unsuitable to system scalability because it requires ID numbering and has bus-traffic limitation. Therefore, in this study for self-reconfiguration, only local IR communication through connected surfaces is used. This enables the modules of a homogeneous system to use exactly the same program without an ID number. Global bus communication is still used in loading a program simultaneously to all modules, and for monitoring multi-program execution and communications.

2.2 Two-Dimensional Self-reconfiguration Task The self-reconfiguration task for this study is an extension of the former one in [2]. In the former experiments, a rectangular structure of various sizes containing 8–24 modules advanced along a fixed direction by a parallel distributed control. The new task is intended for autonomous control with robustness and scalability. Autonomous control implies that both the motion direction

408

H. Kurokawa et al. Before ascent process

A

Walker-pair after ascent process

Anchor

B

four-modulecluster (a) Base structure

(b) Walker-pair and anchors along the travelling path

Fig. 2 Base structure and anchor

and the total structure shape change depending either on the intended task or on obstacle sensing [5]. Robustness and scalability implies that the same controller works with systems of different sizes, and that the system size can vary in real time by adding new modules or by separating faulty modules. Figure 2 portrays processes and terms of the task. All modules are initially assembled in a symmetric planar structure, as shown in Fig. 2(a), hereinafter called a base structure. The base structure can be regarded as constructed by four module clusters. All modules are in P0 posture in Fig. 1(b). Upon a message or an event, two modules on the perimeter of the base structure start an ascent process (A in Fig. 2(a)), and climb on the base structure (B in Fig. 2(b)). These two modules after the process have their joint axes all vertical, and move along the base structure. This pair is called a walkerpair. For a walker-pair to move on the base structure, some modules of the base structure along the path change their postures from P0 to P1 following the sequence in Fig. 1(c), and make their connection surfaces available for the walker-pair to connect as in Fig. 2(b). These upward surfaces are called anchors. A descent process is the reverse of the ascent process, which makes a walker-pair at the perimeter move down and join to the base structure. Algorithms and control programs for the task are organized in layers, from higher decision-making ones to lower hardware control processes. For the upper layers, the base structure is modeled as square cells, as described in the next section. Using such a planar lattice model, studies of several tasks have been performed with distributed algorithms, such as formation of a globally defined shape [8] and parallel motions of flow [6]. Both algorithms are based on vector fields, which assign a motion vector for each cell.

3 Cell Model 3.1 Cell and Symmetry A base structure has symmetry of a square lattice and is represented by square cells, as portrayed in Fig. 3. Four sides of a cell correspond to four out

Toward Flexible and Scalable Self-reconfiguration of M-TRAN

409

Anchor

(a)

Anchor

(b)

Fig. 3 Cell representation. A base structure (a) is represented by a cell pattern (b). The dark triangle in (b) represents that the module is facing its flat surface upward as the anchor in (a)

of six connection surfaces of a module. The correspondence of the surfaces depends on a module’s position and orientation and is represented using a set of symmetric transformations. Each cell has a vector indicating the motion direction assigned by the decision-making process. The two right triangles composing a cell represent semi-cubic parts of a module. Dark colored triangles show corresponding modules forming anchors (Fig. 3(b)). The two semi-cubic parts of an M-TRAN module are different mechanically and logically; only surfaces of opposite polarity can mutually connect. This polarity for connection is automatically satisfied when modules are properly assembled in the base structure; the polarity difference is represented by another symmetric transformation. One more symmetric transformation turns the whole structure upside down, which does not change the cell representation. This transformation is ignored, because it does not take place even locally through self-reconfigurations for the current task.

3.2 Anchor Formation For the walk process, the walker-pair’s motion and the next anchor position are determined according to a decision-making process. An anchor must be prepared before a walker-pair moves a step along the base structure. Successive anchors are at distances of three triangles. Two methods can be used to make and arrange anchors: stationary and dynamic. A stationary method is such that all anchors are initially made over the base structure, as shown in Fig. 4(a). A dynamic method is such that the current anchor connecting a walker-pair requests a nearby cell to become an anchor before one walking step, and the previous anchor to retrieve after the step. For example, when the previous and current anchors are A and B in Fig. 4(b) and when the direction of movement is rightward, the next anchor is selected among candidates indicated by circles. The former study used the stationary method, but this study uses a dynamic one because it allows more surfaces to be connected, thereby causing smaller alignment error.

410

H. Kurokawa et al. 1

1 1 1

(a)

1

2

1

2

2

1

2 1

2

1

2

0 1

2

1

2

1

1

1

0 1

(b)

B

2 1

1 A

0

0

1

1 2

1

1 2

2

1

0

1

1

1

0

1

1 1

2

2

2

2

2

2

2 1 1

1

1

1

2

1

Fig. 4 Stationary anchor pattern (a) and dynamic formation (b). Bold lines represent connected surface pairs and a number in each triangle is that of connected surfaces

3.3 Connectivity and Collision Avoidance The actual module follows the sequence in Fig. 1(c) and releases one or two connections to move its anchor part when a cell becomes an anchor. This release of connections must be considered carefully to avoid fragmentation of the whole. Maintaining total connectivity is generally a difficult problem, especially for a distributed program. We instead introduce allowed local static patterns and transition orders around each node, as shown in Fig. 5. To avoid fragmentation, simultaneous motions of neighbor cells must be inhibited. In addition, when a walker-pair moves, motions of surrounding cells and nearby walker-pairs should be inhibited to avoid collision. For these purposes, cells are grouped dynamically by temporary masters; slave modules are controlled by their masters.

1 2 (a)

2

2

(e)

2

1

1

1 1

1 2

1

2

1

(b)

(c)

NG

2

1

1

0

2

2

1

1

1 1

1

1

1

1 1 (g)

1 1

1 (f)

1

1

1

(d)

1

1

Node

1

1

1

1 NG

0

1

1

1

1

1

0 1

1

1

1

(h)

0 0 1

1

(i)

Fig. 5 Allowed anchor patterns and transitions around a node. Transitions with NG are inhibited. The transition from (f) to (g) must be made by setting one anchor at a time

Toward Flexible and Scalable Self-reconfiguration of M-TRAN

411

3.4 Communication The local anchor patterns presented above are determined also for the sake of communication. Gaps drawn in white in Fig. 5 correspond where two modules are not connected. For that reason, they cannot communicate directly. Two possible methods can be used to ensure communication: managing a path of communication, and always using broadcasting. The former is used here, which is based on a four-cell grouping, as in the figures. Four cells around the node are connected for each pattern in Fig. 5 except pattern (c), and there is one gap at most. Therefore, any two of the four cells can communicate by forwarding a message either clockwise or counter-clockwise around the node. Consequently, four cells for each vertex of an anchor cell are grouped. This grouping is also used for collision and separation avoidance described above. The exceptional pattern, (c) in Fig. 5, is mostly avoided by the upper decision-making layer.

4 Decision-Making and Algorithm Decision-making is done globally or in a distributed manner to set a direction vector to each cell of four possible directions. Various tasks of selfreconfiguration include formation of a globally defined shape in Fig. 6(a), and parallel motions of flow in Fig. 6(b). The figures portray possible direction vectors for these tasks. A vector field like these direction vectors can be made generally based on a dynamic system such as cellular automata or fluid dynamics. The decision-making for the current system, however, is not so simple. The salient difference between this cell model and well-studied square lattice-type modular systems is that two cells cooperate as a walker pair in this system, and two cells of their neighbors help the ascent / descent process. Similarly to the anchor formation in Section 3.2, two methods can be used for this cooperation: static and dynamic. Static clustering is such that all the cells are paired as in Fig. 6(c). This figure, however, illustrates the difficulty in this clustering. Even for the task of a stationary motion along fix direction (left half of Fig. 6(b)), clustering cannot be uniform because the width of this base structure is an odd number. A temporary, non-uniform vector field is necessary as in Fig. 6(c), and the pattern of clustering is not unique to the same original vector field in (b). The principle of dynamic clustering is that only cells at vertices are clustered, as in Fig. 6(d). This dynamic clustering can be more difficult than the above static one, if it aims consistent clustering based only local conditions. In addition to clustering, it is another problem of decision-making when and where an ascent process is initiated. Not only collision avoidance must be considered; the order is also important, e.g., initiation of ascent process at A must be made before that of B in Fig. 6(c).

H. Kurokawa et al. obstacle

412

Initial Final

(b) Field of flow

(a) Shape formation A

D

G

H

I

L

B E

J

F

K

C

(c) Stationary clustering

(d) Dynamic clustering

Fig. 6 Field of flow and clustering

Because of these problems, dynamic clustering was used, and width of the base structure was set an even number in the following simulations and experiments.

5 Software We have been developing programs for simulation and an onboard control. The simulation program is intended as an emulator of the controller, so that most of the controller program runs on the emulator without modification of the source code. An emulator is very helpful for development and debugging, especially such a distributed program. For this purpose, a virtual machine was used in former studies [2]. However, in this study, source codes in C language are made compatible. This compatibility is realized using the multi-thread management with a widely used operating system (Windows OS; Microsoft Corp.). The programs have three layers. The upper two layers are source-code compatible. The top layer is for decision-making, and the middle layer corresponds to processes relating to cell representation. Most processes are implemented as discrete state machines, but an ascent process is a sequence of motions that contains about 50 individual motion steps using six cells. This ascent process uses token passing and is programmed reversible. The reversibility will be useful for a future study so that modules will be able to change their states or retrieve their motions always, even during the ascent process. The bottom layer is for communications by CAN and IR and for physical properties. As for the onboard controller, programs for joint angle control, connection control, IR communication and proximity sensing, etc. are implemented in the four CPUs of each module. The simulation program has

Toward Flexible and Scalable Self-reconfiguration of M-TRAN

(d)

(e)

413

(f)

Fig. 7 Experiments: (a)(b)(c) – A single walker-pair traveled from one end to another; (d)(e)(f) – Non-uniform vector field

emulator functions for these mechanical functions of individual modules and kinematics simulation of the whole, as well as 3D display by OpenGL, and management of threads on Windows OS. In addition, states of cells are visualized as presented in Fig. 8. Currently, only the top layer distributes direction vectors according to global coordinates. Nine major states are defined such as MASTER, WALKER, ANCHOR, SLAVE, IDLE, and minor ones for such information as the direction vector, clustering, grouping, obstacle sensing, and connective relation with others. For the bottom layer of the controller, 17 states are used for connection control, either to ensure successful connection, to make retrial in case of failure, or to initiate connection process from a passive (female) side module by sending an IR message to an active (male) side module. Possible events or inputs to a cell include receiving an IR message, detecting obstacles, and completion of motions of joint angle control or connection control. Allowed actions of a cell are state transitions, most of which are followed by either IR message transmission or mechanical controls. The numbers of message types are 14 in the middle layer and 25 for connection in the bottom layer. For the preliminary experiment using eight modules in Figs. 7(a)–7(c), in which a single walker-pair moved over the base structure, 266 messages were transferred in total, including 139 for cell dynamics and 127 for connection. Messages containing either 1-, 2-, or 3-byte data are used; sending 2-byte data required about 100 ms because of a low bit rate (333 bps) and a redundant protocol. The speed, however, is not important for this study.

6 Experiments and Discussion Simple experiments resembling those depicted in Fig. 7 were carried out, yielding good results. Self-reconfiguration motion and their exchanged messages mostly coincided with simulation results except differences caused by

414

H. Kurokawa et al.

Zone 2

Zone 2

1 Zone 1

4

1 (a) Initial base structure 1

(b) Cell model of (a)

2

(c)

3

4

2

3 5 5

4 3

(d)

6

(e)

(f)

7

6

7 5 8 5

7

6

8 (g)

9 (h)

(i) Modules corresponding to (g)

Fig. 8 Simulation snapshots for a vector field composed of two uniform fields in zones 1 and 2. The initial base structure is an 8 × 4 rectangle in (a) and (b). Decision making by dynamic clustering described in Section 4 and a cluster flow motion are illustrated by numbered boxes containing two cells with or without arrows. The numbers show clustering order and boxes with arrows show clusters whose ascent process are initiated

the speed of motions and communication or by retrial of connection. Currently, self-reconfiguration for larger structures and non-uniform vector fields works well in simulations (Fig. 8), but not perfectly by experiments. We need not only to make emulation more precise but also to improve the algorithm in the upper layer. For motion videos, see [9]. The most serious problem that arose in experiments was alignment error. The position and/or orientation error between two surfaces must be within a certain range (about 2–3 mm) for successful connection. Because the apertures of IR transmitters and detectors are small, similar alignment error causes failure of IR communication when two surfaces are not connected. In experiments, both the connecting motion and communication were repeated to recover such failures. Retrials took place several times during the experiments. Many functions remain to be implemented. First, power management is necessary for a larger experiment for longer duration. Although most of the base structure is stable and rigid, all modules retain active joint control. Such control is power-consuming and can be switched off according to each module’s state. Secondly, obstacle sensing has not been integrated into the control program, although it has been verified by other experiments. It should be

Toward Flexible and Scalable Self-reconfiguration of M-TRAN

415

implemented to make the system autonomous. Finally, the upper layer programs should be extended. Current interaction between layers is restricted so that the system’s behavior is deterministic and simply analyzed. The second layer originally has various possibilities of design in modules’ motions, in walker motions, and in anchor selection, as illustrated in Fig. 4(b), to yield identical or similar results in the cell model, but deterministic and fixed decisions were made. Such decision-making in the top and second layers should be naturally more flexible for the system’s robustness and behavior diversity.

7 Remark Scalable self-reconfiguration by M-TRAN was described in this paper. The system was modeled using a discrete state machine based on square cells; both simulation and control programs were developed. Experiments for simple selfreconfiguration by more than a dozen modules were performed. Though a systematic design of algorithms and programs were aimed, each layer of the developed program was not designed simple enough. Especially for the top layer, results of other algorithmic studies for lattice modular systems were not directly applicable to the self-reconfiguration task of the M-TRAN modules, and further studies will be necessary. Moreover, though the M-TRAN system is intended for three-dimensional self-reconfiguration, experimental results and remaining hardware problems suggest that only two-dimensional structures supported by a flat surface are suitable for scalable self-reconfiguration, and truly three-dimensional selfreconfiguration may be out of reach for the current hardware. However, results and a knowledge by this work will be the bases for studies toward development of truly three-dimensional self-reconfiguration using improved design modules.

References 1. Kurokawa, H., Murata, S., Yoshida, E., Tomita, K., Kokaji, S.: A 3-D SelfReconfigurable Structure and Experiments. In: Proc. IEEE–RSJ International Conference on Intelligent Robots and Systems (IROS 1998), pp. 860–865 (1998) 2. Kurokawa, H., Tomita, K., Kamimura, A., Kokaji, S., Hasuo, T., Murata, S.: Distributed Self-reconfiguration of M-TRAN III Modular Robotic System. Intl. J. Robotics Research 27(3-4), 373–386 (2008) 3. Jorgensen, M.W., Ostergaard, E.H., Lund, H.H.: Modular ATRON: Modules for a self-reconfigurable robot. In: Proc. IROS 2004, pp. 2068–2073 (2004) 4. Kurokawa, H., Tomita, K., Kamimura, A., Kokaji, S., Terada, Y., Murata, S.: Distributed Metamorphosis Control of a Modular Robotic System M-TRAN. In: Distributed Autonomous Robotic Systems (DARS), vol. 7, pp. 115–124. Springer, Heidelberg (2006)

416

H. Kurokawa et al.

5. Ostergaard, E.H., Tomita, K., Kurokawa, H.: Distributed Metamorphosis of Regular M-TRAN Structures. In: Distributed Autonomous Robotic Systems (DARS), vol. 6, pp. 169–178. Springer, Heidelberg (2004) 6. Butler, Z., et al.: Generic Decentralized Control for Lattice-Based SelfReconfigurable Robots. Intl. J. Robotics Research 23(9), 919–937 (2004) 7. Varshavskaya, P., Kaelbling, L.P., Rus, D.: Automated Design of Adaptive Controllers for Modular Robots using Reinforcement Learning. Intl. J. Robotics Research 27(3-4), 505–526 (2008) 8. Stoy, K., Nagpal, R.: Self-Repair Through Scale Independent SelfReconfiguration. In: Proc. IEEE-RSJ Intl. Conf. Intelligent Robots and Systems (IROS 2004), pp. 2062–2067 (2004) 9. http://unit.aist.go.jp/is/dsysd/mtran3/index.htm

Reconfigurable Teams: Cooperative Goal Seeking with Self-Reconfigurable Robots Zack Butler and Eric Fabricant

Abstract. Self-reconfigurable (SR) robots have been developed with the promise of enabling a wide range of abilities within a single hardware system. Particularly in the context of locomotion, SR robots may be able to traverse a wider range of environments than traditional legged or wheeled robots. Additionally, with a large number of modules, the system may be able to divide to explore areas in parallel and recombine to make larger, more capable groups. However, the choice of how to divide and merge these “reconfigurable teams” to most effectively solve tasks such as exploration of unknown terrain remains unanswered. This exploration problem can be seen as a superset of traditional multi-robot exploration, as robots must not only choose places to visit but also may coordinate to dynamically adjust the number of entities in the team. In this paper, we present a state-based distributed control algorithm for entities within a reconfigurable team that uses local sensory information and a team model to choose actions within the environment. We show a set of empirical results from simulation that show how the best choice of strategy may depend on the nature of the environment.

1 Introduction Self-reconfigurable (SR) robots are systems composed of a group of modules which can change their relative topology to achieve different tasks. For example, a group of modules can construct a tower to position a sensor high above the ground, but then reform into a long narrow snake shape to maneuver through a narrow opening in a wall. In this work, we are interested in Zack Butler and Eric Fabricant Department of Computer Science Rochester Institute of Technology Rochester NY USA e-mail: {zjb,erf9559}@cs.rit.edu

418

Z. Butler and E. Fabricant

the idea that a system with a large number of modules may be able to form several independent groups, and work as a team of mobile robots, but with new and interesting capabilities. We introduce the term reconÞgurable teams to suggest that not only is the overall system composed of self-reconfiguring modules which can make different shapes as the situation warrants, but that the entire team of robots can be reconfigured to have more or fewer members with variable size and capability during task execution. This can allow tasks such as exploration to be completed in parallel when possible while still retaining the ability to form larger, more capable robots as necessary. Previous work in SR robotics has generally proceeded from the bottom up. Several different hardware systems have been developed, followed by centralized reconfiguration control and distributed control for both locomotion and reconfiguration. The first proposal was CEBOT [5], which included fairly complex physically cooperative robot modules, but following systems concentrated on simpler homogeneous modules, both lattice-based (in which modules fill space densely) [15, 9] and chain-based (in which links of modules can move effectively without reconfiguration) [16, 3]. More recent hardware such as MTRAN [10] and Superbot [13] can take on the characteristics of either type. Working from a biological viewpoint, the Swarm-Bots project [6] has developed a mobile robot design capable of self-assembly and disassembly for physical tasks such as hole-crossing, while retaining completely independent motion by each robot module, unlike most SR systems. We note that although the algorithms presented here are inspired by our work with lattice-based robots, they are designed to be general enough to be applied to any of these systems. Control algorithms for SR systems have primarily focused on two problems: reconfiguration, or how to generate a particular desired shape or topology of modules; and locomotion, or how to move a group of modules across their environment. Some early hardware-specific reconfiguration algorithms were proposed by Vona and Rus [12] and Yim et al. [17]. Locomotion for chain-based systems has been described by Zhang et al. [18] and Shen et al [14]. More generic algorithms for locomotion of lattice-based systems have also been described [4]. We have also previously described ways in which a group of lattice-based modules can split into two components and recombine with no centralized control in the context of locomotion [2]. Our work here implicitly assumes that these problems have been solved for the particular hardware system being used. That is, we look a group of SR modules as a single mobile entity which can navigate across its environment (within given physical constraints) and split and merge when desirable, and we are developing techniques to leverage these capabilities. This work is also quite similar to multi-robot exploration, in which a team tries to visit and map an unknown environment under distributed control. Previous work in this area tends to concentrate on maximizing the amount of map area discovered rather than explicit goal locations, and tends to use more fine-grained cooperation than our technique [1, 7]. Our technique also

Reconfigurable Teams: Cooperative Goal Seeking with SR Robots

419

does not create a full map as in most previous work, but instead relies on local sensing of the environment. It is also partially inspired by the ALLIANCE architecture for general multi-robot task allocation [11], in which robots have different tasks to perform, but can become impatient and either give up their task or move to assist another robot that is not completing its job. We similarly use a state-based architecture and have robots timeout if they cannot complete their task, though we also use explicit calls for help when a robot becomes stuck. Finally, biologically-inspired approaches to multi-robot foraging and exploration tasks have also been proposed (e.g. [8]). Our work is similar in its reactive nature, but we use more explicit messaging between entities than most of these techniques (for example, to claim a particular goal and direct others elsewhere rather than cooperation based on pheromones or implicit cooperation via appropriate control laws). The ability to split and recombine to change the structure of the system also requires a more complex decision-making process.

2 Problem Description In this work, we are developing distributed algorithms for exploration in the context of reconfigurable teams. Rather than focusing on the motions of individual modules, which can vary significantly from one hardware implementation to another, we assume that the system consists of groups of modules with inherent locomotion capability. We use the term blob to denote such a group of modules acting as a member of a reconfigurable team, and blob recon
urations to mean those actions in which either a blob splits into two smaller blobs or two smaller blobs merge into one larger one. Traditional reconfigurations will likely also occur within a blob to perform locomotion or other tasks, however these actions are not explicitly considered in this work. As a domain focus, we are investigating the problem of exploration of an unknown environment with a team of self-reconfiguring robots. In particular, we consider the situation of an initially unknown environment with a set of predefined locations (goals) that need to be visited. This implies that the blobs share a global coordinate system that they can maintain their location within. More general exploration can be implemented simply by placing goal locations densely in the environment. We implement here only the coordination at the blob level, not at the individual module level, though coordination at that lower level is also important. Our goal is to develop distributed algorithms under which each blob can decide what goal to attempt to reach, as well as whether or not it should split (to achieve two goals in parallel) or merge with another blob (to form a blob with greater capability). There is a spectrum of ways to approach this problem: with reactive control, with search-based planning, or with some intermediate approach blending the two. In a reactive approach, each blob chooses its action based on its immediately available information, which may include both sensor data

420

Z. Butler and E. Fabricant

and information about the states of other blobs. The planning approach involves searching all possible action sequences over a map (including both motion as well as splits and merges) to come up with optimal choices for future actions. Our work is toward the reactive end of this spectrum, where decisions are made based on current local information with only some localized path planning. This allows for much simpler computations and lower resource requirements on the modules, as detailed maps do not have to be maintained and searched, though potentially at the expense of completeness in the exploration process. In the course of this work, we have abstracted the physicality of the SR system to focus more clearly on the problem of deciding how to apportion resources to the exploration tasks at hand. However, the specific allocation depends critically on the ability of the hardware. Thus, we define a set of system parameters that describe these capabilities. In particular, we define a maximum slope value slmax , such that any group is assumed to be able to climb any slope no steeper than that, and a step height stmax as the height (measured in module diameters) that a blob can surmount regardless of slope. The idea here is that a lattice-based robot can form a tower to climb a relatively short cliff, while a chain-based robot may also be able to raise some of its modules in a self-supported structure. We also define a sensing radius rs within which the blob can detect the topography of its environment. For any of these parameters, we allow that they may depend explicitly on q, the number of modules in the blob (we also use Q to denote the total number of modules in the system). They may also be constant — for example, chain-based systems that use cantilevered modules would have stmax limited by motor strength rather than q. Specifically, our experiments use values inspired by our previous work in simulations of lattice-based systems: slmax = 45◦ , stmax = q/3, and rs = 3qr ,  3 where qr = 2q/3π is the radius of a hemisphere with volume q. Of course any values can be used for these parameters depending on the intention of the simulation. Finally, we assume that all blobs can communicate with each other throughout the environment through wireless means.

3 Algorithm Design As mentioned previously, we have chosen a primarily reactive distributed algorithm to control the blobs within our system. Our algorithm bases its decisions on the blob’s sensor data about its immediate neighborhood along with one internal state and the reported states of the other blobs in the system. The system architecture is divided into two components, primarily for clarity of exposition. These are the Navigation-Communication Unit (NCU), which is responsible for the overall decision making, and the Local Navigation Unit (LNU) which does short-range path planning (within the blob’s sensor radius) at the request of the NCU. These components are assumed to run as a single instance within each blob, either in a leader module or collectively.

Reconfigurable Teams: Cooperative Goal Seeking with SR Robots

421

Fig. 1 State machine of the NCU. States outlined in black are communicated to the team, grey states are not

3.1 Local Navigation Unit The LNU is provided with three inputs to determine a good local path for the robot. The first of these is the global coordinate of the intended destination (d), which is provided by the NCU. The second and third inputs are 2rs × 2rs matrices of discretized environmental measurements, centered at the blob’s position. Essentially, these matrices are grids that correspond to the section of the environment that a blob can perceive. One matrix consists of the local terrain’s elevation values and the other their corresponding slopes. In our implementation of the LNU, square grids are used as a generalization of what a blob can sense to keep the sensory perimeter simple. The purpose of the LNU is then to plan a navigable path to a point within the sensor radius that is as close as possible to d. A list of all points in the local grid are first sorted by favoring those closer to d. For each of these points, starting with the one closest to d, the shortest path from the blob’s location to that point is computed and then tested for traversability according to the blob’s physical limitations. That is, each sequence of points along the path with slope greater than slmax constitutes a step, and so the height difference along that step is added together. If this total is greater than stmax , then this path is not navigable and the next possible local destination will be tested. If it is not (or there is no point along the path with a slope > slmax ), then that local destination is returned to the NCU.

3.2 Navigation Communication Unit The NCU operates by controlling a simple state machine that determines the overall behavior of the blob. It takes a list of goals as initial input and

422

Z. Butler and E. Fabricant

maintains a list of goal locations not yet reached by the team. It also monitors incoming messages from other blobs and maintains an active model of the team’s state; when a blob signals the team of its state change, the remainder of the team use the notifier’s identifier, status, and location to update the model. Specifically, the NCU keeps track of each blob’s most recent intended destination, as well as which blobs cannot navigate in their desired direction due to the local topography. The list of goals and team model are used to produce its decision (and potentially an internal state change). As the NCU constantly references its model and sensor input, it may be in any of the seven internal states shown in Fig. 1. These are primarily in a loop of choosing a destination and informing the team, navigating toward it, and either reaching it or getting stuck. Along the way to a destination, it generally remains within the “navigating” state and does not reconsider its choice, though it may split itself along the way as described below. It uses the LNU to direct it toward the selected destination through intermediate points as long as possible. If at any point the LNU reports that there is no way to approach the desired destination, the blob transitions to the “stuck” state. It will then remain at its location as long as it knows that another blob is attempting to approach it. When no blob is actively en route to assist the stuck blob, the stuck blob will begin its timeout sequence. Upon timing out it will attempt to pursue a different destination to reduce the inefficiency of having its modules be inactive for prolonged periods of time. When choosing a destination, the NCU is faced with decisions of whether to choose a goal location, split to attempt to achieve multiple goals, or combine with another blob to help the other blob to achieve its goal. In general, there is not necessarily a correct (or computably correct) choice amongst these possible actions, especially with a reactive algorithm. Without complete knowledge of the environment, it is not possible to say a priori whether it is faster to wait for another blob to surmount an obstacle or try a different goal and leave the closer goal for future exploration. With this in mind, we have developed and tested several different basic behaviors to see how the use of different heuristic choices impacts efficiency and reliability in different environments. In the “generous” behavior, the NCU is configured to give priority to blobs that are stuck over goal points, and will choose the closest such blob. Assisting blobs that are stuck increases the number of active modules on the team, improving the team’s overall abilities by decreasing the likelihood of any given member becoming stuck. We allow multiple blobs to help a single immobile blob simultaneously, providing a degree of redundancy for unsticking in case one or more blobs are too small, whereas goal points are only pursued by one blob at a time because the blob can always request help if it needs it. We also tested a “greedy” behavior, in which the NCU will always choose the closest unclaimed goal location as long as one exists before helping any stuck blobs. This may be appropriate if the merge action is difficult, as the time taken to merge is not explicitly considered in our algorithm. Finally, we also tested a “neutral” behavior, in which the NCU chooses the

Reconfigurable Teams: Cooperative Goal Seeking with SR Robots

423

closest potential destination, whether a stuck blob or a goal location, and a “limited” behavior in which splits and merges are not considered at all, to examine how much advantage is obtained from blob reconfigurations. A blob may choose to split into two smaller blobs to improve parallelism throughout the mission. In order to ensure useful splitting, we define a lower bound on the group size ql , such that each resulting blob must contain at least this many modules. This avoids producing blobs that are likely to get stuck, and since traversability depends on blob size, the value of ql should ideally depend on the complexity of the environment, if known. In this work, we have chosen ql such that the smallest blob can surmount a step of 1/10 the total elevation change of the environment. If this constraint can be satisfied, there are two times that a blob may split. First, it may split after reaching its destination and before choosing a new destination. Second, while navigating, if an alternate destination (i.e. an immobile blob or goal point) is within the blob’s sensory radius, the LNU is capable of assessing what size blob is required to reach that point. If the total blob is sufficiently large, it will split such that one of the resulting blobs is just large enough to reach the visible destination. This approach conserves modules for the original destination while ensuring that the visible destination can be reached. In both cases, both blobs resulting from a split contain the same the full team model known prior to the split.

4 Experimental Results In order to evaluate the utility of our algorithms, we performed a suite of experiments while varying different independent parameters, both environmental and algorithmic. The goal is to determine under what circumstances blob reconfigurations can be beneficial, either in terms of improved efficiency or more reliable goal attainment. We chose a set of environments, initial team sizes and configurations, and which of the algorithmic behaviors was used by the blobs, and tested all combinations of these parameters. For each set of parameters, we then measured the success rate (number of goals achieved) as well as the total travel distance and number of blob reconfigurations taken to achieve the goals. Total travel distance is measured as the average distance traveled by a module in the system, and for most systems indicates the total energy consumed during the mission. To compute this, the distance traveled by a blob during its lifetime is weighted by the size of the blob and the total over all blobs is divided by the team size Q. To perform these experiments, we developed a graphical simulator for reconfigurable teams. The simulator spawns a separate thread for each blob to inject some of the non-determinism inherent in a real system, and uses an additional thread to handle communication between the blobs. Four different environments were developed for testing, as displayed in Fig. 2. All the environments were 500× 500 module diameters in size. Three

424

Z. Butler and E. Fabricant

Fig. 2 Environments used for testing: (A)-(C) are increasingly more challenging versions of the same terrain, while (D) is constructed to require cooperation while allowing simple navigation through the majority of the environment. In our testing, goal locations were placed randomly for each run in environments (A)-(C) while in environment (D) one goal location was placed atop each pillar Table 1 Results from simpler environment (Fig. 2A) for different team sizes Q. Initial configurations are four groups near the corners of the environment (::), three groups in a triangle near the center (△) and one large group in the center (•). Each entry in the table represents the average of 25 trials behavior / Splits | Merges | Average module travel init-config Q = 100 Q = 200 Q = 400 Q = 800 generous/:: 0 | 0.52 | 846 0 | 0.04 | 691 0 | 0 | 707 4 | 0 | 446 generous/△ 0 | 0.44 | 1181 0 | 0.04 | 1037 0 | 0 | 983 3 | 0 | 626 generous/• 0 | 0 | 2639 1.04 | 0.04 | 1428 1.08 | 0 | 1412 7.04 | 0 | 663 greedy/:: 0 | 0.56 | 798 0 | 0 | 686 0 | 0 | 711 4.04 | 0.04 | 439 greedy/△ 0 | 0.24 | 1078 0 | 0 | 987 0 | 0 | 1021 3.04 | 0 | 597 greedy/• 0 | 0 | 2617 1.16 | 0.08 | 1495 1.12 | 0 | 1485 7 | 0 | 668 neutral/:: 0 | 0.6 | 759 0 | 0 | 684 0 | 0 | 684 4 | 0 | 431 neutral/△ 0 | 0.68 | 1161 0 | 0 | 978 0 | 0 | 1004 3.04 | 0 | 574 neutral/• 0.08 | 0.04 | 2612 1 | 0 | 1527 1.04 | 0.04 | 1527 7 | 0 | 681 limited/:: 0 | 0 | 739 0 | 0 | 746 0 | 0 | 705 0 | 0 | 700 limited/△ 0 | 0 | 1045 0 | 0 | 1005 0 | 0 | 1005 0 | 0 | 980 limited/• 0 | 0 | 2568 0 | 0 | 2600 0 | 0 | 2572 0 | 0 | 2700

of them (Fig. 2A-C) have the same general shape, with a total elevation change of 255 module diameters but with increasing jaggedness that calls for larger blobs. That is, the easiest map was smoothed such that most slopes

Reconfigurable Teams: Cooperative Goal Seeking with SR Robots

425

Table 2 Results from more challenging environments. Goals remaining are out of 30 initial goals for the first two environments and out of 4 for the Pillars environment behavior / init-config generous/:: generous/△ generous/• greedy/:: greedy/△ greedy/• neutral/:: neutral/△ neutral/• limited/:: limited/△ limited/•

Splits | Merges | Avg. module travel | Goals unreached Medium slope High slope Pillars 4.6 | 6.56 | 1996 | 0.76 5.96 | 8.16 | 1959 | 7.4 1.1 | 3.8 | 1491 | 0.4 3.32 | 4.6 | 1983 | 2.84 7.24 | 8.76 | 2702 | 8.04 1 | 3 | 1064 | 0 7.08 | 6.2 | 2053 | 5.28 12.12 | 11.48 | 2722 | 6.04 2.2 | 2.0 | 2032 | 0.5 0.12 | 1.44 | 993 | 1.44 0.44 | 1.56 | 1436 | 3.6 0.9 | 3.8 | 1488 | 0.2 0.32 | 1.64 | 1525 | 0.44 0.32 | 0.68 | 1705 | 7.04 1 | 3 | 1562 | 0 3.28 | 1.52 | 1115 | 4.2 3.8 | 1.48 | 1728 | 8.4 2 | 2 | 1267 | 0 1.08 | 2.84 | 1160 | 1 1.8 | 3.4 | 1164 | 4.32 1 | 3.9 | 1286 | 0.2 1.28 | 2.28 | 1804 | 2.28 1.84 | 2.68 | 1560 | 6.2 1 | 3 | 901 | 0 4.96 | 3.92 | 1557 | 0.92 5.96 | 4.72 | 2188 | 6 2 | 2 | 1230 | 0 0 | 0 | 813 | 1.28 0 | 0 | 1087 | 8.84 0 | 0 | 7437 | 3 0 | 0 | 1061 | 1.8 0 | 0 | 2041 | 11.04 0 | 0 | 4995 | 3 0 | 0 | 2616 | 2.2 0 | 0 | 2065 | 7.72 0 | 0 | 1112 | 0

were less than slmax , while the most difficult had steps requiring nearly all modules to surmount. In each of these maps, 25 trials were run for each set of parameters, with 30 goal points placed at random locations for each trial. A fourth environment (Fig. 2D) is primarily flat but with tall pillars at each corner that need to be explored, requiring 100, 200, 300 and 400 modules to climb. For this case we used hard-coded goal locations, with one goal at the center of each pillar. In each environment, we tested three distinct initial configurations for the modules: in four groups near the corners of the environment, in three groups in a triangle close to the center of the environment, and in one large group in the center. On the easiest map (Fig. 2A), it was possible for the reactive technique to achieve all goal locations, and so we can compare the amount of work required under each robot size, initial configuration and behavior. The results are shown in tabular form in Table 1. In general, as expected, very few merge operations were required, since blobs were unlikely to get stuck, though some were used when the initial blob sizes were small (e.g. 100 modules divided among four groups). This has the effect that any of the self-reconfiguring behaviors will have substantially the same behavior, since there will be little need to be generous or greedy. On the other hand, with larger numbers of modules, splits did generally occur, helping the overall efficiency of the exploration. If the modules start in several groups, then the distance traveled is similar across the different behaviors, but if they start in one group (as would be a common deployment), then the use of splits dramatically decreases the distance that the modules have to travel. On the more difficult terrains (Fig. 2B-C), due to the larger cliffs, we primarily tested with teams of 400 total modules. Here we can see how the different behaviors effect the reactive algorithm’s ability to reach all desired

426

Z. Butler and E. Fabricant

locations. Perhaps surprisingly, the generous algorithm is in fact the least effective overall, using significantly more effort to achieve about the same or fewer goals. Our observations of these simulations are simply that the blobs spend too much effort attempting to help each other over the more challenging terrain and not enough going to reachable goals first. On the other hand, the greedy behavior does quite well, and in fact still shows a number of split and merge operations; it is simply that goal-directed behavior takes precedence and the reconfigurations allow for additional goals to be achieved in an implicit second phase. The neutral behavior generally takes more energy to reach a slightly larger number of goals than the greedy behavior, especially on the medium-difficulty environment (Fig. 2B). In the pillar environment (Fig. 2D), the reactive algorithm is put at somewhat of a disadvantage, since it will optimistically split before reaching the pillars and finding that it needs to merge again. When the initial group is large enough to climb all the pillars, the limited behavior is most efficient, but this behavior cannot reach three of the four goals unless it starts in a single large group. This environment seems to favor the neutral behavior that balances merges with goal locations to best form groups that can climb each pillar.

5 Discussion/Future Work Overall, the use of blob reconfiguration in the context of reconfigurable teams has shown benefits in a variety of different situations. In particular, in environments that are more easily traversable, it allows any deployment to effectively explore in parallel, while in more challenging situations, the best behaviors can allow the system to collectively reach more goal locations more efficiently than without blob reconfigurations. The neutral behavior, in which destinations are chosen based purely on proximity rather than whether the destination represents a blob or a goal location, had the best overall success and efficiency, though a greedy behavior can be more efficient in difficult environments at the expense of achieving somewhat fewer goals. One intriguing avenue of further study is the use of more complex mapbased algorithms instead of the simple reactive LNU used here. This should allow the blobs to reach more of the goals in the environment, but requires the modules to store and share large amounts of information as they explore. In the extreme case, with complete information, this becomes a superset of the multiple traveling salesmen problem (mTSP would define the “limited” behavior problem), which is NP-hard but has approximation algorithms available. However, we are still most concerned with cases where the initial environment is largely unknown. Using planning in a partially known environment will also require tighter integration of planning with the higherlevel decision making to effectively choose long-term actions, since a blob will have to judge whether to take a potentially roundabout route to a goal or ask for help to make a larger blob that can take a more direct route.

Reconfigurable Teams: Cooperative Goal Seeking with SR Robots

427

Finally, we also plan to extend this work to handle heterogeneous SR systems. In such systems, modules have specialized capabilities, such as different sensors, and so blobs will be specified by their qualitative abilities as well as the quantitative parameters used in this work. Then, as in multi-robot task allocation, goals can be tagged (either a priori or after an initial detection) with the type of sensor that needs to visit that location, and the blobs will have to not only choose their goal location but also to split and merge with their sensor capabilities in mind.

References 1. Burgard, W., Moors, M., Fox, D., Simmons, R., Thrun, S.: Collaborative multirobot exploration. In: IEEE International Conference on Robotics and Automation, pp. 476–481 (2000) 2. Butler, Z., Murata, S., Rus, D.: Distributed replication algorithms for selfreconfiguring modular robots. In: Distributed Autonomous Robotic Systems, vol. 5, pp. 37–48 (2002) 3. Castano, A., Shen, W.-M., Will, P.: CONRO: Towards deployable robots with inter-robots metamorphic capabilities. Autonomous Robots 8(3), 309–324 (2000) 4. Fitch, R., Butler, Z.: Million module march: Scalable locomotion for large selfreconfigurable robots. Int’l. Jnl. of Robotics Research 27(3-4), 331–343 (2008) 5. Fukuda, T., Ueyama, T.: Cellular Robotics and Microrobotic Systems. World Scientific, Singapore (1994) 6. Groß, R., Bonani, M., Mondada, F., Dorigo, M.: Autonomous self-assembly in swarm-bots. IEEE Trans. on Robotics 22(6), 1115–1130 (2006) 7. Howard, A., Kitchen, L.: Cooperative localisation and mapping. In: Proc. of the 1999 Int’l. Conf. on Field and Service Robotics (1999) 8. Kube, C.R., Zhang, H.: Collective robotic intelligence. In: Proc. of the Second Int’l. Conf. on Simulation of Adaptive Behavior, pp. 460–468 (1992) 9. Murata, S., Kurokawa, H., Yoshida, E., Tomita, K., Kokaji, S.: A 3-D selfreconfigurable structure. In: Proc. of IEEE ICRA, pp. 432–439 (May 1998) 10. Murata, S., Yoshida, E., Kamimura, A., Kurokawa, H., Tomita, K., Kokaji, S.: M-TRAN: Self-reconfigurable modular robotic system. IEEE/ASME Trans. on Mechatronics 7(4), 431–441 (2002) 11. Parker, L.E.: ALLIANCE: An architecture for fault tolerant, cooperative control of heterogeneous mobile robots. In: Proc. of IEEE Int’l Conf. on Intelligent Robots and Systems, Munich, pp. 776–783 (September 1994) 12. Rus, D., Vona, M.: Self-reconfiguration planning with unit compressible modules. In: Proc. of IEEE ICRA, pp. 2513–2520 (1999) 13. Shen, W.-M., Krivokon, M., Chiu, H., Everist, J., Rubenstein, M., Venkatesh, J.: Multimode locomotion for reconfigurable robots. Autonomous Robots 20(2), 165–177 (2006) 14. Shen, W.-M., Salemi, B., Will, P.: Hormone-inspired adaptive communication and distributed control for conro self-reconfigurable robots. IEEE Transactions on Robotics and Automation (October 2002) 15. Suh, J., Homans, S., Yim, M.: Telecubes: Mechanical design of a module for self-reconfigurable robotics. In: Proc. of IEEE ICRA, pp. 4095–4101 (2002)

428

Z. Butler and E. Fabricant

16. Yim, M., Duff, D., Roufas, K.: PolyBot: a modular reconfigurable robot. In: Proc. of IEEE ICRA, pp. 514–520 (2000) 17. Yim, M., Zhang, Y., Lamping, J., Mao, E.: Distributed control for 3D metamorphosis. Autonomous Robots 10(1), 41–56 (2001) 18. Zhang, Y., Yim, M., Eldershaw, C., Duff, D., Roufas, K.: Scalable and reconfigurable configurations and locomotion gaits for chain-type modular reconfigurable robots. In: Proceedings of the International Symposium on Computational Intelligence in Robotics and Automation, pp. 893–899 (2003)

“Deformable Wheel”-A Self-recovering Modular Rolling Track Harris Chi Ho Chiu, Michael Rubenstein, and Wei-Min Shen

Abstract. The rolling track is an effective modular robot configuration with high maneuverability. However, one technical barrier that prevents it from practical usage is that most existing rolling track robots must start from a typical stand-up position and they are also difficult to turn while rolling. This paper presents a solution for these problems. By extending our previous work, we have developed a set of new gaits for the rolling track to self-standup, roll and turn. The combination of these behaviors on a single SuperBot rolling track has enabled it to roll from any initial conditions, steer while rolling, and recover from falling sideways. These new gaits have been demonstrated with 6 modules in hardware and 8 and 10 modules in simulation. In addition, the new gaits can be activated even reconfiguring from a snake configuration.

1 Introduction Unlike a wheel, a modular rolling track propels forward by actively changing its shape. A common problem experienced by the wheel and rolling track is the inability to perform self-recovery, meaning that it cannot stand up without any external help once it has fallen sideways. An example would be a rolling track recovering from a flattened orientation as in Figure 1(a) to a stand-up posture as in Figure 1(b). A rolling track can be deemed ”complete” provided it is able to self-recover, roll and turn such that the robot is able to traverse the environment. However, current research has been limited to demonstration of rolling behavior in modules with different mechanical designs and analysis of rolling gaits. Matsuda et. al.[3]controls the stiffness of the joints between modules to change shape for rolling forward. Other Harris Chi Ho Chiu, Michael Rubenstein, and Wei-Min Shen Information Sciences Institute, The University of Southern California, 4676 Admiralty Way, Marina Del Rey, CA 90292 e-mail: {chichiu,mrubenst}@usc.edu, [email protected]

430

H.C.H. Chiu et al.

(a) Fall sideways

(b) Stand-up posture

Fig. 1 6-Module SuperBot Rolling Track

methods involved rotary or prismatic joints change their angles based on time [9][12] or sensor feedback[11][2][5]. In terms of speed analysis of rolling gaits, Sastra et. al[5] presented an analysis on sensor-based rolling and recorded a fast speed of rolling (1.4m/s) on CKBot. These controls are required to start from a particular stand-up posture. Notice that their rolling motion is restricted to a single plane. The degrees of freedom and the strength of actuators only contribute to motion along the direction of travel. To perform turning and self-recovery, motions along a perpendicular plane is essential. An extra degree of freedom along a perpendicular plan allows the rolling track to steer to a different direction during rolling and lifting adjacent modules upward for self-recovery. In particular to self-recovery, the actuators providing the extra degree of freedom have to be strong enough to lift up adjacent modules while avoiding tipping over during self-recovery process. Previous work by Yim[12] has simulated rolling and turning while Shen[8] has demonstrated self-recovery in simulation. However, a control algorithm containing self-recovery together with turning and rolling has neither been proposed nor tested in hardware. The work presented here contributes by providing the control method for self-recovery, turning and finally combines these with rolling to form a “complete” rolling track demonstrated for the first time on a modular robot. The control method is scalable in number of modules and adaptable to topology changes in modular robots. The paper is organized as follows: Section 2 introduces our hardware platform. Section 3 presents a rolling track detects its loop configuration. Section 4 presents the control used for self-recovery. In Section 5, we introduced a algorithm for turning built on top of rolling motion. Section 6 shows an integration of self-recovery, turning and rolling together with configuration detection. Experiment results are shown in Section 7. Section 7.1 demonstrates the “complete” integration using a remote controlled 6-module rolling track for rolling, turning and self-recovering. In Section 7.2, rolling tracks of size 6, 8, and 10-module have been tested. Self-recovery and rolling behaviors are shown after the loop of rolling track is completed from a chain (snake) configuration in Section 7.3. Section 8 concludes the paper with discussions and future research directions.

“Deformable Wheel”-A Self-recovering Modular Rolling Track

431

Fig. 2 A single module with local coordinate frame for gravity vector Table 1 Comparison of current rolling track implementations d.o.f. SuperBot[4] PolyPod[12] CONRO[9] MTRAN[2] CKBot[5] 5RChain[11] BIYOn[3]

3 2 2 2 1 1 1

extra d.o.f. not for rolling motion 2 1(limited range) 1 1(connected differently) 1(connected differently) 0 0

Turning

Self-Recovery

Yes Yes Yes Maybe Maybe No No

Yes No No Maybe No No No

2 Constructing the Rolling Track Table 1 shows the comparison of current implementations of rolling track using different modular robots. Only some are capable to do turning and most of them are unlikely to do self-recovery. PolyPod is limited by its joint angle range for self-recovery. MTRAN and CKBot are possible to turn only if they are connected differently with a 90◦ rotation to adjacent module. However, it requires 12 or more modules to form a feasible rolling track and requires each motor to provide enough torque to lift 3.5 times of module weight. Such configuration of rolling track might not roll in practice. SuperBot modules[4] are used as our experiment platform for its flexibility of 3-degree-of-freedom (d.o.f) and its ability to load two neighboring modules in each degree of freedom. Figure 2 shows a SuperBot module and its 3 d.o.f. (Pitch, Roll, Yaw) are highlighted. Each module also equips with a build-in 3D accelerometer for detection of its own orientation by knowing the gravity vector relative to its local coordinate frame. Therefore, SuperBot is a favorable choice to implement “complete” behaviors of rolling track.

432

H.C.H. Chiu et al.

Table 2 RULEBASED Table for Configuration Detection Type Recv Hormone Action Send Hormone FT & BK NOTALOOP Reset timeout forward hormone BK n/a n/a (NOTALOOP, ranNum) FT NOTALOOP n/a n/a Each connection type sets isInLoop=false upon receiving NOTALOOP hormone

Our implementation connects 6 SuperBot modules together with the FRONT interface connected to the BACK interface of another module to form a loop. There are total 6 interfaces namely, F RON T , BACK, LEF T , RIGHT , U P and DOW N . Each interface can communicate to a connected neighbor through Infra-Red sensors. Modules are either connected autonomously through genderless connectors [7] or manually mounted with screws. As shown in Figure 2, each SuperBot module is of dimension {length, width, height}= {2k, k, k}, where k = 6 inches. For the rest of the paper, the joint configuration pitch, roll, yaw of SuperBot shown in Figure 2 has a value of {0, 90, 0}and is treated as default. Positive pitch indicates a clockwise rotation while positive Yaw indicates anti-clockwise rotation. For Roll axis, the joint configuration becomes {0, 180, 0}if the right half of the module is rotated 90◦ anticlockwisely about x-axis. In the 6-module rolling track, the joint angles of all 6 modules are {30, 90, 30}. For describing the use of gravity vector, we define the local coordinate frame as show in Figure 2 with z-axis pointing inward. We also have directly adopted the asynchronous algorithm described in section 4.4.1 by Itai et. al.[1] to determine the size of rolling track before the use of any control algorithms.

3 Configuration Detection Modular robots can formed into various topologies by connecting other modules in different ways. In previous work, loop topology is always assumed and motions for rolling track are designed based on this assumption. Therefore, it is essential for modules knowing whether they are a part of a loop before running any controls on a rolling track and detect topology changes in case of reconfiguration or module failure. We extend the identifier-free Adaptive Communication (AC) protocol by Shen et. al.[10] to detect loop formation. Each module discovers its connection type using AC protocol by listening to probe messages periodically sent from neigboring modules. For example, the connection type of a module is FRONT (FT) if it has received probe messages only from its FRONT interface. Similarly, the connector type is FT and BACK (BK) if probe messages are received from its FRONT and BACK interfaces. Each module looks up a pre-programmed RULEBASED table (Table 2) to act accordingly based on its connection type. For BK connection type, the module generates a NOTALOOP hormone message

“Deformable Wheel”-A Self-recovering Modular Rolling Track

433

Table 3 Pre-programmed table of joint angle {pitch, roll, yaw} in degree for selfrecovery. (Values changed from previous step are in bold)

Step Step Step Step Step Step Step Step Step

1 2 3 4 5 6 7 8 9

hopCount hopCount hopCount hopCount hopCount hopCount 0 1 2 3 4 5 {30,90,30} {30,90,30} {30,90,30} {30, 90, 30} {30, 90, 30} {30, 90, 30} {0,90,0} {20,0,90} {90,180,20} {0, 90, 0} {20,0,90} {90,180,20} {0,90,90} {55, 0,90} {90,180,55} {90, 90, 0} {20, 0, 90} {90,180, 20} {90,90,90} {0,0,90} {90,180,0} {90, 90,90} {0, 0,90} {90,180,0} {90,90,90} {0,0,0} {0,180, 0} {0, 90,0} { 0, 0,0} {0,180, 0} {90,90,55} {35,90,35} {35,90,35} {55, 90,-90} { 0, 0, 0} { 0,180, 0} {65,90,55} {35,90,35} {35,90,35} {90, 90,90} {-70, 0, 0} { 0,180,55} {65,90,55} {35,90,35} {35,90,35} {90, 90, 90} {-70,90, 0} { 0 ,90, 55} {30,90,30} {30,90,30} {30,90,30} {30, 90, 30} {30, 90,30} {30, 90,30}

embeded with a random number sends to its BK connector. The use of random number is to prevent looping of the hormone message upon the loop is completed. The message will then continue to propagate through module with connection type FT and BK. If the loop is open, the hormone message will terminate at the one with FT. If the loop is closed, no NOTALOOP message will be generated as there is no BK connection type. To handle topology changes, each module also has a countdown timeout variable notInLoopTimeout. If a module is in FT and BK and no NOTALOOP hormone message has been receieved in the timeout period, the module can declare it as part of a loop and a flag isInLoop = true. If a NOTALOOP hormone message has been received, notInLoopTimeout will be reset and isInLoop = false. This timeout mechanism adds flexibility to loop detection as isInLoop will not be made permanent and dynamically follows topology changes.

4 Self-recovery Control The self-recovery reconfiguration procedure requires synchronization between modules that can be achieved using leader-based control. In a loop formation, leader is elected dynamically through probabilistic leader election[1]. Each module contains the same pre-programmed look-up table of joint angles(See Table 3) and therefore every module has to know the total number of module in advance. Angles can be looked up based on current reconfiguration step and hopCount from the leader in the recovery sequence. Leader starts with sending out a message ”step=0, hopCount=0”. Each module passes the modified message ”step, hopCount+1” to the next module. Module can look up the joint angles in the table with corresponding step and hopCount. The control terminates when the leader receives a message with hopCount equal to the number of module at the final step. The self-recovery is separated into 3 phases - Initiate, Fold-Up-and-Expand, and Resume. In the 6-Module Superbot rolling track configuration (6M-Loop),

434

H.C.H. Chiu et al.

(a) Step 1

(b) Step 2

(c) Step 3

(d) Step 4

(e) Step 5

(f) Step 6

(g) Step 7

(h) Step 8

(i) Step 9

Fig. 3 A 6-module SuperBot rolling track recovers from falling down

the rolling track first initializes to be a regular hexagon shape as seen in Step 1 of Figure 3. With the Initiate phase, the subsequent steps can always remain the same regardless of any hexagon shape it is previously. Step 2 to 5 are of Fold-Up-and-Expand phase. The aim is to use 3 degrees of freedom (Pitch, Yaw, Roll ) to change from configuration having dimension {length, width, height}- {6k,2k,k}in step 2 to {6k,k,2k}in step 5. Three modules are lifted up in step 3 and the rolling track is folded as a block of size {3k,2k,2k}in step 4 and spanning horizontally in step 5. In the Resume phase, the rolling track has to “unwind” the twist at roll axis of 4 modules done in step 2. The “unwind” can be refered to the change of roll joint angle in step 6 for hopCount = 1, 2 and in step 8 for hopCount = 4, 5 in Table 3. It then resumes regular hexagon configuration in step 9. Table 3 shows the look-up table used in each module of 6-Module Superbot rolling track to perform recovery.

5 Turning While Rolling Turning a rolling track into a different direction requires steering during rolling motion. The steering has to be performed just right before the module lands on the ground or it will discontinue the rolling motion or it causes the rolling track to fall sideways. Therefore, the design of control for rolling has to have turning in mind. We extend our previous work [8] on dynamic rolling by adding turning to the rolling control. Figure 4 shows a simulation of 6-module Superbot rolling track (6M-Loop) rolls forward by changing its shape to move the center of gravity forward. Every time a rolling track has a horizontal orientation as in

“Deformable Wheel”-A Self-recovering Modular Rolling Track

435

(a) Gravity vector relative to the (b) Rolling track changed shape local frame for moving center of gravity Fig. 4 6-module rolling track changes its center of gravity

Figure 4(a), it changes the shape to a squeezed hexagon shape as shown in Figure 4(b). The unbalance causes rotation about the bottom module and thus the rolling track tips over and rolls forward. The motions repeat in cycle triggered by the orientation of modules. Turning can be done with a slight “twist” at the roll joint before a module touching the ground. Our implementation is based on reactive control mapping the orientation of each module to a set of joint angle values (Pitch, Roll, Yaw). We consider the case for n number of modules, for each module i, accelerometer values are directly mapped to joint angles as in (1). For a polygon-shaped rolling track in a up-right posture, each module will have unique orientation and hence different range of accelerometer values. motor anglesi = f (acc values range)

(1)

In relating shape to joint angle, we consider n is even and denote the set of modules to be module0 , module1 , ..., modulen−1 such that modulei´s BACK connector is connected to module(i+1)%n´s FRONT connector. For any modulei , its joint angles are denoted as (pitchi , rolli , yawi ). In a n-sided polygon shape, each internal angle of the polygon is governed by αi = 180◦ − (pitchi +yawi ). If the internal angle of the tip of the squeezed n-sided polygon is θ, then for n-Module closed-loop configuration, the joint angles become: For all i = n/2 − 1 or n − 1, (pitchi , rolli , yawi ) = ((180◦ − θ)/2, 90◦ , (180◦ − θ)/2)

(2)

For all other i, (pitchi , rolli , yawi ) = (θ/(n − 2), 90◦ , θ/(n − 2))

(3)

To have the rolling track in squeezed polygon shape realizing its orientation as in Figure 4(a), values have to be obtained from the accelerometers so that a new cycle of shape changing can be invoked. To simplify the procedure

436

H.C.H. Chiu et al.

in obtaining values for every module, we can measure the orientation - angle of gravity vector to local coordinate frame. For modulek , if angle βk is the gravity vector relative to its local frame, the angle of gravity vector for adjacent module can be obtained using, β(k+1)%n = βk − αk

(4)

An example is shown in Figure 4(a) with n = 6. Red arrows represent gravity vector and same as Figure 2, black arrows represents local coordinate frame. If we have obtained angle of one red arrow to its local frame in modulek , the angle of red arrow of modulek+1 can be calculated by subtracting its internal angle αk . Then, the mapping from (1) can be done by translating accelerometer values into angle of gravity vector to its local frame and corresponding joint angle can be obtained by using (2) and (3) with small tolerance on the angle of gravity vector index k for modulek is incremented anticlockwisely and position dependent. That means any module in the same orientation will be having the same index and index will always start from left module first touching the ground as show in Figure 4(a). Turning of the rolling track can be achieved by changing the roll joint of module1 and module2 when they are about to touch the ground. By changing their roll angle roll1 = 90 − γ and roll2 = 90 + γ for some γ, the rolling track will be able to turn. To counter balance the upper part of the rolling track, module in diagonal position can be set to have roll joint angle in counter direction rolln/2+1 = 90 + γ and rolln/2+2 = 90 − γ. Experiment results of rolling and turning are presented in Section 7.

6 Integration of “Complete” Behaviors on a Rolling Track To have “complete” behaviors on rolling track, controls for self-recovery, turning, rolling and also configuration detection have to be coordinated properly. Pseudocode 1 shows how the behaviors are intergrated by mode switching. Operation modes including rolling mode (ROLL MODE ), turning mode (TURN MODE ), self-recovery mode (RECOVERY MODE ) and idle mode (IDLE MODE ) are switched based on current topology, orientation and remote commands. The rolling mode is activated after a loop topology is realized through configuration detection in Section 3. Rolling mode and turning mode can be switch interchangeably using remote command. In any case the rolling track fall sideways, self-recovery mode is activated and recovery sequences will be carried out. At the end of the sequence, the rolling track will be switched back to rolling mode. In case of any topology changes, if rolling track are no longer in a loop, it will fall back to idle mode.

“Deformable Wheel”-A Self-recovering Modular Rolling Track

437

Pseudocode 1. Mode Switching in Distributed Control for Rolling Track mode = IDLE_MODE Loop: configurationDetection()//isInLoop= true if the module is in a loop if(isInLoop) switch(mode) case ROLL_MODE: if(orientation == fall sideways) mode = RECOVERY_MODE if(remoteCommand == TURN) mode = TURN_MODE case TURN_MODE: if(orientation == fall sideways) mode = RECOVERY_MODE if(remoteCommand == ROLL) mode = ROLL_MODE case RECOVERY_MODE: if(RecoveryStep >= LAST_STEP) mode = ROLL_MODE case IDLE_MODE: detectLoopSize(); mode = ROLL_MODE else mode = IDLE_MODE EndLoop

7 Experimental Results 7.1 Remotely Controllable Rolling Track Rolling, turning and self-recovery are implemented on a remote controllable 6-module SuperBot rolling track to show “complete” behaviors. In the implementation, the speed of rolling during turning is decreased by reducing the

(a) Rolling track turns (b) Rolling track is made (c) Rolling track resumes right to the door way flattened rolling after self-recovery Fig. 5 Snapshots of the experiment performing self-recovery, turning and rolling behaviors

438

H.C.H. Chiu et al.

(a) 6-module track (b) 8-module track (c) 10-module track Fig. 6 SuperBot rolling track of size 6, 8 and 10 in simulation

strength of the motor to lower the possibility to fall sideways. We discover that the transition from forward rolling gait to a turning gait failed occasionally if the switch command is not issued remotely in time during falling forward. Then, a scenario of the “complete” behaviors has been carried out without turning off the power or reloading the program. At the beginning, the rolling tracks rolls forward at the same speed (about 0.35m/s) as reported in our previous work[6]. The robot is then placed flatten down and recovered successfully by carrying out the recovery steps every 3 seconds as shown in Figure 3. The rolling track starts rolling and it is commanded to turn using remote control. Figure 5 shows the self-recovering, turning, and rolling motion during the experiment. Video of the experiment can be found at the following website: http://www.isi.edu/robots/superbot/movies/rcRollingTrack.avi

7.2 Scalability for Rolling and Turning The proposed control of rolling and turning is scalable in number of modules. Each module selects its joint angle based on its accelerometer values from its own unique orientation in a polygon, therefore, identifier is not required. The control also requires no message exchange avoiding hop delay issue. Rolling track of size 6-module, 8-module and 10-module are demonstrated in simulation. They are implemented in SuperBot simulation using Open Dynamic Engine. Control programs have been loaded into simulated modules without any modifications to the control program. As shown in Figure 6, the rolling tracks of different size are able to detect its configuration and turn while rolling. The experiment suggests the algorithm can support a higher number

(a) initial 6-module (b) intermediate (c) intermediate snake step 1 step 2

(d) 6M-Loop

Fig. 7 Self-Reconfiguration from a 6-module snake to 6M-Loop rolling track

“Deformable Wheel”-A Self-recovering Modular Rolling Track

439

of modules if the joints are strong enough to support its load for rolling motion. Video of 6-module, 8-module, 10-module rolling track simulation can be viewed at: http://www.isi.edu/robots/superbot/movies/rtSimRolling.avi (or rtSimRolling.swf for faster download)

7.3 Reconfiguration Experiment in Simulation As SuperBot is designed to be self-reconfigurable, we would like to test the future adaptability of the control to loop formation. In Figure 7, a preprogrammed reconfiguration procedure is implemented to form a 6-module rolling track from a 6-module snake configuration in simulation. Every module starts with same control algorithm. Upon topology changed to a loop, the loop is detected and the size of loop is calculated by the loop verification algorithm proposed in Section 3 and self-recovery is then activated once fall-down orientation is detected. The mode switching in Section 6. It starts to roll forward again. Video of simulation on a 6-module docking, fall-down recovery and dynamic rolling can be viewed at: http://www.isi.edu/robots/superbot/movies/rtSimReconfig.avi (or rtSimReconfig.swf for faster download)

8 Conclusion and Future Work This paper addressed the problem of self-recovery of a rolling track and documented the first implementation of self-recovery on a moduler robot. It also provided “complete” control algorithm combining gaits performing rolling, turning and fall-down recovery on a remote-controllable Superbot rolling track. The potential to support more number of modules for rolling and turning are examined in simulation of 6, 8, 10-module configuration. Furthermore, the ability of the control to support self-reconfiguration is demonstrated with the use of loop and ring size detection in a snake to loop reconfiguration. Future research directions include addressing problems caused by distributed control such as motor conflicts and instability of gait transition from rolling to turning, attaining higher speeds and generalizing self-recovery reconfiguration procedures for Superbot. We believe regulating motor torque using current sensors will reduce the motor conflicts. Acknowledgements. We are very grateful that the SuperBot project is sponsored by NASA Cooperative Agreement NNA05CS38A. We would like to thank Duckho Kim and Dr. Behnam Salemi for tuning and maintenance of SuperBot module and the works of our colleagues at Polymorphic Robotics Laboratory at the Information Sciences Institute of the University of Southern California in carrying out the experiments. We would also like to thank Dr. Mark Moll for building a framework on SuperBot simulation.

440

H.C.H. Chiu et al.

References 1. Itai, A., Rodeh, M.: Symmetry breaking in distributed networks. Information and Computation 88(1), 60–87 (1990) 2. Kamimura, A., Kurokawa, H., Yoshida, E., Murata, S., Tomita, K., Kokaji, S.: Automatic locomotion design and experiments for a modular robotic system. IEEE/ASME Transactions on Mechatronics 10, 314–325 (2005) 3. Matsuda, T., Murata, S.: Stiffness distribution control - locomotion of closed link robot with mechanical softness. In: Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006, May 15-19, pp. 1491–1498 (2006) 4. Salemi, B., Moll, M., Shen, W.-M.: Superbot: A deployable, multi-functional, and modular self-reconfigurable robotic system. In: IEEE/RSJ International Conference on Intelligent Robots, Beijing, China (October 2006) (to appear) 5. Sastra, J., Chitta, S., Yim, M.: Dynamic rolling for a modular loop robot. In: International Symposium on Experimental Robotics, Rio de Janeiro, Brazil (2006) 6. Shen, W.-M., Chiu, H., Rubenstein, M., Salemi, B.: Rolling and climbing by the multifunctional superbot reconfigurable robotic system. Albuquerque, New Mexico (February 2008) 7. Shen, W.-M., Kovac, R., Rubenstein, M.: Singo: A single-end-operative and genderless connector for self-reconfiguration, self-assembly and self-healing. In: 2008 IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems, Nice, France (September 2008) 8. Shen, W.-M., Krivokon, M., Chiu, H., Everist, J., Rubenstein, M., Venkatesh, J.: Multimode locomotion for reconfigurable robots. Autonomous Robots 20(2), 165–177 (2006) 9. Shen, W.-M., Salemi, B., Will, P.: Hormone-inspired adaptive communication and distributed control for CONRO self-reconfigurable robots 18(5), 700–712 (October 2002) 10. Shen, W.-M., Salemi, B., Will, P.: Hormone-inspired adaptive communication and distributed control for conro self-reconfigurable robots. IEEE Trans. on Robotics and Automation 18(5), 700–712 (2002) 11. Tasuku, Y., Osamu, M., Tooru, O.: Dynamic rolling control of 5r closed kinematic chain with passive joints. Journal of the Robotics Society of Japan 22(1), 112–119 (2004) 12. Yim, M.: Locomotion with A unit-modular reconfigurable robot. Technical Report CS-TR-95-1536 (1995)

Exploit Morphology to Simplify Docking of Self-reconfigurable Robots Kasper Stoy, David J. Christensen, David Brandt, Mirko Bordignon, and Ulrik Pagh Schultz

Abstract. In this paper we demonstrate how to dock two self-reconfigurable robots and as a result merge them into one large robot. The novel feature of our approach is that the configuration we choose for our robots allows the robots to handle misalignment errors and dock simply by pushing against each other. In 90 experiments with the ATRON self-reconfigurable robot we demonstrate that two three-module robots can dock in 16 seconds without using sensors and are successful in between 93% and 40% of the attempts depending on approach angle and offset. While this is a modest step towards fast and reliable docking, we conclude that choosing appropriate configurations for docking is a significant tool for speeding up docking.

1 Introduction In nature, slime molds of the dictyostelids group search for food and divide individually, however, once their bacterial food source in the local area is exhausted they assemble into a slug that allows for faster locomotion and thus enable the slime molds to reach a new area faster[11]. It is this idea we aim to transfer to robotics. E.g. we may want individual robots to search an area of a collapsed building for survivors and upon completion merge into a large robot with greater mobility that efficiently can move to the next building. One potential way to reach this goal is to use self-reconfigurable robots. A self-reconfigurable robot is a robot built from modules that can move around on each other automatically and thus allows the robot as a whole to change Kasper Stoy, David J. Christensen, David Brandt, Mirko Bordignon, and Ulrik Pagh Schultz Modular Robotics Group, The Maersk McKinney Moller Institute, University of Southern Denmark, Campusvej 55, DK-5230 Odense M, Denmark e-mail: {kaspers,david,david.brandt,mirko,ups}@mmmi.sdu.dk

442

K. Stoy et al.

shape. In the field of self-reconfigurable robots much attention has been paid to self-reconfiguration both in chain-type robots and in lattice-type robots. In lattice-type robots the lattice helps modules align and thereby makes it possible to perform self-reconfiguration open-looped. In chain-type robots, on the other hand, self-reconfiguration is more difficult because modules need to use their typically limited number of sensors to locate and dock with other modules. The self-reconfiguration ability of lattice-type robots is superior when they are in a lattice, but this advantage disappears when two unconnected, lattice-type robots have to dock because the robots no longer can assume that their lattices guarantee alignment since they may approach each other from unknown angles. In this case lattice-type robots like chain-type robots have to move towards, align with, and dock with each other. This is the problem we address in this paper. Docking of two separate self-reconfigurable robots has been documented before as we will discuss in the related work section, but the novelty of our approach is the way that we choose the configuration of the two robots to significantly simplify the docking process. In contrast to other works in this area, we let the robots approach each other with their broad sides facing each other. We choose the shapes of the robots so that once the robots are close enough and aligned to a certain degree they can just push towards each other and rely on the mechanical structure to perform the final, precise alignment. In our implementation sensors are only used to detect if another robot is present, but since the robots’ broad sides face each other sensors are distributed over a larger area and makes it easier to make an initial, rough alignment of the robots even with few imprecise sensors. In experiments we demonstrate that two robots can dock modestly fast and reliable relying on their shape alone. In 90 experiments with the ATRON selfreconfigurable robot we demonstrate that two robots can cover a distance of 30cm, perform an alignment behavior, and dock on average within 16 seconds. The success rate of docking depends on the angle and offset between the two robots. If they are initially aligned they are successful in 94% of the docking attempts, however, the success rate depends on angle and for the worst angle they are only successful in 40% of the attempts. At long-range the shapes of the robots are of little use and therefore sensors are in most cases needed to get the robots within short-range of each other. However, by exploiting the configuration to aid docking less precise sensors can be used because the shape of the robot can compensate for the imprecision. Alternatively, exploiting the configuration can complement a sensor-only approach and increase the speed of docking.

2 Docking in Self-reconfigurable Robots Self-reconfigurable robots were originally envisioned by Fukuda [3]. The physical realization of his vision, CEBOT Mark I, II, and IV, were what we today

Exploit Morphology to Simplify Docking of Self-reconfigurable Robots

443

IR Sensors

Fig. 1 The two-wheeled robot (right) used in the experiments is built from ATRON modules (left). A rubber ring is attached to each of the side modules so they can work as wheels and one of the connectors of the front module is slightly extended to work as a castor

describe as mobile multi-robot systems since each module had wheels and the robots were not in practice able to self-reconfigure in three dimensions. However, interestingly enough the docking process he used also relied on matching shapes, but it was two individual robots that docked as opposed to groups of modules as it is in our case. Self-reconfigurable robots come in three types: chain-type, lattice-type, and hybrid. Chain-type robots, such as CONRO [2] and PolyBot [16], are robots that as the name suggests are built from chains of modules that are connected to form a robot. This type of robot is particularly well suited for locomotion. In lattice-type robots, e.g Fracta [6] and Crystalline [14], modules are organized in a lattice structure similar to atoms in a crystalline lattice which facilitates easier self-reconfiguration. Most modern self-reconfigurable robots, however, are of the hybrid type and are able to change between a chain form and a lattice form. Examples include the M-TRAN robot [7], the MoleCube robot [20] and the ATRON robot [9], which we will use for our experiments. In other multi-robot systems docking has also been investigated for instance in the context of the Swarm-Bots multi-robot system [4, 5] and in the work by Bererton et al.[1], but in both cases docking is based on vision. In chain-type robots a self-reconfiguration step is similar to docking except that it is internal to one robot. A common approach to self-reconfiguration is to use inverse kinematics [18] or simpler angle considerations [15] to bring the modules which are about to dock within sensor range of each other. Upon detecting each other the two modules start a docking process. This process is typically guided by infrared transmitters and receivers [12, 10] and sometimes accelerometers [19]. The docking of two robots has also been studied before. Rubenstein et al. [13] show how two CONRO robots can approach and dock. However, in contrast to our approach the configuration of these robots is not exploited to simplify docking. The same is also true for the following approaches, however, they distinguish themselves by using

444

K. Stoy et al.

a

b

d

e

c

Fig. 2 The docking sequence. a) The two robots approach each other - the male to the left and the female to the right. b) Once the male detects the female it drives into it. c) The female and male robot then reverses, the male slightly slower than the female. After a few seconds they stop. d) the female then moves slightly forward to get in perfect position to connect. e) the female extends its two connectors and the two robots are docked

rich sensors such as a small on-board camera. Yim et al. [17] use a system where diodes on a module blink in a predefined way that allows another module using an on-board camera to calculate an approach vector and move accordingly. In their robot magnets are used for the final alignment and docking. Murata et al. also use vision to guide the docking process, but use a special docking configuration that is able to clutch a docking robot [8]. This approach is similar to ours, but in our approach there is no internal movement of the configurations and the two docking robots play a more equal role. Our perspective is also different since our goal is to exploit the shape of the robots to allow docking without the use of advanced sensors like vision.

3 The Docking Behavior The most important aspect of our approach to docking is the morphology of the robots. While the particular configuration we use here does not transfer directly to other self-reconfigurable robots the idea of exploiting the configuration does. E.g. it possible to build two complementary shapes that facilitates easier docking. In our implementation we use the ATRON self-reconfigurable robot [9]. An ATRON module is a self-contained robot in its own right. It has onboard power and computation, the two hemispheres from which it is made can rotate independently, and it can communicate with neighbors using infrared communication. Furthermore, it can connect to a female connector of

Exploit Morphology to Simplify Docking of Self-reconfigurable Robots Fig. 3 This figure shows the relationship between the distance between the center of rotation of a robot and the sensor it uses to control its alignment

445

derr

sensor r


err Center of rotation

a neighbor module by extending its male connector of which there are two on each hemisphere. The other two being female. In our experiments we use the three-module configuration of ATRON modules shown in Figure 1. The two modules on the sides can rotate independently and function as wheels providing the robot with a high level of mobility. The shape of the robots is such that if one robot is rear-ended by another robot the shape of the robots helps them align. In our implementation infrared communication is only used to detect if another robot is there or not. However, the configuration we have chosen allows us in the future to increase robustness of our approach by using sensors. The idea is that the distance between the infrared sensors, positioned almost at the extremes of the wheels, and the center of rotation is relatively large. This means that a large error in distance sensing does not translate directly into a large alignment error. Rather it is reduced by a factor equal to the distance from the sensor to the center of rotation. Let us assume that the distance sensor is position at a distance r from the center of rotation as illustrated in Figure 3. The error in distance derr that this sensor measures is related to the error in alignment θerr by:   derr (1) θerr = arctan r The important thing to note here is that the further the sensor is from the center of rotation, that is, the larger the r the smaller the error in alignment. As an example consider our two-wheeled robot, which has a radius of 5cm, and assume a distance measurement error of 1cm. In this case the alignment error is then 11◦ degrees, for which the shape of the robots may be able to compensate. Let us continue by describing our docking process in details (also refer to Figure 2). Both robots start in search mode. In our implementation the robots just move forward because the focus is the docking process, but more useful search behaviors can be implemented. Once the robots detect each other the rear robot goes into female mode and the front one into male mode. They then have to align themselves for the actual docking to take place. In our approach this is done by a small sequence of steps. When the male robot, coming from the rear, detects the female using its infrared communication channel it drives into the female to ensure that it is aware that the male is there. After this the male slowly reverses while the female robot follows

446

K. Stoy et al.

Head

Docked

Idle

Wheel

msg rcv

Timeout

Male Wait

Male Slow Reverse

other robot detected

to both wheels

msg rcv

Search

Init

Search

to head msg rcv

Docked

other robot detected

to other wheel

msg rcv

Female Wait

Female Reverse

msg rcv

Timeout

Fig. 4 All modules use the same control system, an overview of which is shown here. All modules start in the init state and then depending on their local connection topology decides to become either a head (left) or wheel (right). The circles represent states, solid arrows state transitions, boxes conditions for performing the corresponding transition, and finally dashed arrows indicate a message being passed. Note, that there are two wheels in a configuration of which only one is shown here

it and aligns only relying on the shape of the female and the male robots. Once this behavior has been active for a few seconds the male stops and the female reverses into it and relies on the shapes of the two robots to handle the final, precise alignment. The final step is a simple, timed process that moves the female robot forward a bit to get it in perfect position for extending its connectors and connecting to the male and thus completing the docking process. After docking is complete, to make experimentation easier, the robots disconnect and return to search mode. The docking process is intuitively quite simple, but the controller still becomes a little complicated because we, instead of controlling two mobile robots, have to control six independent modules. All modules have identical controllers, but depending on their connection topology they take on different roles. In each robot, one module becomes head while two become wheels. The

Exploit Morphology to Simplify Docking of Self-reconfigurable Robots

447

coordination of these three modules is straight forward, but still requires some message passing. The main idea is that the head module controls the search for another robot by controlling the wheels, which are slaves. If the head detects another robot in front of it the robot enters male mode. If, on the other hand, one of the wheels detects a robot behind it the robot enters female mode. Once these modes have been established control is transferred to the wheels that go through an open-looped docking sequence that exploit the robots shape to align them. If only one robot detects the other it still goes through the docking sequence, but of course docking fails. An overview of the control system can be seen in Figure 4.

4 Experiments In the experiments we describe here we are only interested in the docking aspect and not the robots’ ability to locate each other or get in position for the docking sequence to start. It is expected to be relatively simple to include these aspects by having the robots randomly explore their environment and upon encountering each other circle each other to get in position for docking. However, we have not implemented this behavior and therefore only report on the docking aspect. In the experiments we place the two robots at a distance of 30cm from each other facing the same direction. The male robot heads towards the female robot fifteen times from four different angles: 0◦ , 11,75◦ , 22,5◦, 45◦ . We also let the male robot approach the female 15 times with an off-set of 0cm, 2cm, and 4cm while maintaining the same heading as the female. The angle and offset are measured as illustrated in Figure 5. We also tried with larger angles and offsets, but the male robot tended to fail to detect the female robot and therefore the docking algorithm didn’t get started. For each attempt we record success or failure and if it is a failure the reason. We identified two types of failures the most serious is connect failure, this occurs when everything went well, but the robots ended up not being aligned to a high enough degree to successfully connect. The second failure is less serious

Offset

Fig. 5 The male robot to the right approaches the female robot from a range of angles and offsets measured as indicated. The reference point is the center of rotation of the female robot

Angle

448

K. Stoy et al.

Table 1 This table shows the number of successful and failed docking attempts as a function of the angle (degrees) between the two robots (left) and the off-set between the robots (right)

Angle Successes Connect failure Tire failure Off-set Successes Connect failure 0 14 1 0 0cm 14 1 11,75 6 9 0 2cm 15 0 4cm 12 3 22,5 10 5 0 45 9 1 5

and occurs when the male upon impact causes the female to loose a tire (the thin rubber rings that work as tires) and thus the female looses the ability to move straight. This is a less serious failure because in theory it can be avoided by attaching the rubber rings more securely. However, in practice this turned out to be difficult. The results of the experiments are summarized in Table 1. Interestingly enough the success of docking seems related to the angle with which the two robots approach each other in an unexpected way. In fact, docking is most successful at angles of 0◦ and 45◦ (disregarding the tire failure). The reason for this is that at medium angles the male robot, after driving into the female, is likely to get stuck in a misalignment where it is locked by the mechanical structure. In other words, there is a local minimum in the shape of the robot which we should consider removing by using another configuration or if needed if we redesign the module. Docking takes approximately 16 seconds including covering the initial distance of 30cm and performing the docking sequence. Specifically, from watching video recordings we estimate that it takes 5 seconds for the male robot to cover the 30cm and drive into the female, 8 seconds to execute the docking sequence, and 3 seconds for the female to move a bit forward and extend its connectors. The time taken to complete each attempt is nearly the same, because the only variable is when the two robots detect each other. This is also confirmed by observation of the experiments. A last thing to note, which cannot be observed from the numbers, is that while a connect failure indicates that the connection failed, it does not imply that the robots missed each other by a large margin. In fact, in all the failed experiments we estimate from observation that the connectors ended up within approximately 1cm of the area where successful connection happen. Therefore, if given a second attempt it may result in a success.

5 Comparison and Analysis The question is now what we have achieved by implementing a docking behavior that exploits the configuration of the female and male robot.

Exploit Morphology to Simplify Docking of Self-reconfigurable Robots

449

One performance metric for docking is time. In our experiments we find that the two robots are able to dock in approximately 16 seconds. We can compare this to the docking time of 5-10min reported by Rubenstein et al. [13] and while it is not reported videos suggest that Yim et al. and Murata et al. achieve docking in a few minutes. This is not surprising since having the robot drive into each other and relying on mechanical structure to perform the alignment should be faster than a more careful, but slow approach where the robots approach each other based on sensor information. On the other hand it should also be expected that the sensor-based approaches are more robust and can handle larger alignment errors. In terms of robustness this is certainly true: both Rubenstein et al. and Murata et al. report 100% success compared to our 92% - 40% depending on angle. However, in terms of alignment error this is not the case, which maybe seem surprising. Rubenstein et al. report that they can handle an alignment error of ± 45◦ and Murata et al. that the female, which is actually the only one doing the sensing, can handle an alignment error of ± 20◦ . This should be compared to the ± 45◦ we report here. The reason is very simple: the robots have to be within the field of view of the sensors before the sensor-based docking behavior can be activated. This leads to an interesting hypothesis: for docking a combination of a sensorbased behavior and a mechanical compliant configuration may result in both reduced docking time as well as increased robustness. The documentation of this hypothesis is work in progress. Let us analyze in more detail what the mechanical compliance actually provided. We measured the area in which the connector mechanism of the modules themselves are able to handle the misalignment. This area has a height of 1cm in the direction orthogonal to the female robot’s heading and a width of 0.85cm in the direction parallel to its heading. The center of this area is located 1.25cm in the direction opposite to the robot’s heading measured from the position where the robots are mechanically aligned and push against each other (as shown in Figure 2c). In other words, a docking behavior that does not exploit the shape of the robot has to control the male robot so it ends up in an area of 0.85cm2 for the connector to successfully connect. However, we saw in our experiments that a docking behavior that exploits the shape extends this area to one that is 8cm wide and ranges from when the robots are pushed completely against each other to when they are far apart. In a similar way, we measured the misalignment angle for which the mechanical connector can compensate to be 6.7◦ . As we saw in the experiments the docking behavior is able to extend this angle to 45◦ with varying degree of success. We can interpret this larger area as the area an additional sensor-based behavior has to be able to bring the male robot into to dock successfully relying on the shape of the robots alone. Experiments, which are currently work in progress, support that implementing such a behavior is not difficult. However, if we on the other hand didn’t exploit the shape the sensor-based behavior would have to bring the robot within the small area in which the connector itself can compensate for misalignment. This would be a difficult

450

K. Stoy et al.

problem whose solution most likely will involve high-resolution sensors. We therefore conclude that it is beneficial to exploit configuration to aid in the final alignment and docking of two robots.

6 Discussion The success of our approach is not due to the specific docking process we chose, in fact, it may be better just to have the male drive into the female until precise alignment has been achieved or some other strategy. It seems that the success is not tied to the design of the ATRON module either. We believe that it is due to us exploiting the morphology of our robots to simplify docking. Although it certainly requires further investigation we feel that exploiting morphology is a principle that may help us develop docking methods which scale with the number of modules, which generalize to three dimensions and which may allow us to dock two robots of arbitrary configurations as we will discuss below. While we have no experiments to support it, we think that our approach may scale to robots that consist of a higher number of modules because we can construct them in a way that makes their morphologies suited for docking. It may, for instance, be possible to connect three of our three-module robots in a triangular shape and build a female to match it. However, the success of this approach depends on the robots’ ability to turn. Another alternative is to have a swarm of three-wheel robots that dock with a larger configuration that reproduce the female morphology proposed here in each layer. In the work reported here we are only considering a two-dimensional case. However, additional modules may also be used to aid alignment in three dimensions, but it is probably not enough to exploit morphology to achieve docking in three dimensions when, for instance, the robots need to dock in rugged terrain. In the ATRON robot and other robots there are on-board tilt sensors that can be used to control tilt and twist. Yaw and the two planar position coordinates can be handled as we have described in this paper. Remaining is the vertical position coordinate, which it seems likely can be controlled in a similar way by using infrared transmitter and receivers and by exploiting the morphology also in the third dimension. The wheel based approach demonstrated here does not work in rough terrain, but by using basic primitives such as forward, backward, turn left, and turn right one can use any configuration which can perform these basic action. In other words, we may be able to disassociate the docking sequence from the actual shape of the configurations as long as we can use a few modules to build a morphology suitable for docking. We have no experimental data to support these thoughts at this point, but will in future work investigate these question and hopefully realize the goal of fast, reliable, scalable, three-dimensional docking of self-reconfigurable robots of arbitrary configurations.

Exploit Morphology to Simplify Docking of Self-reconfigurable Robots

451

7 Conclusion In this paper we addressed the problem of docking of self-reconfigurable robots. We showed that by exploiting the morphology of the robots we can simplify the task of docking significantly. Specifically, we make docking easier by exploiting the shape of the robots and furthermore let them approach each other with their broad sides facing each other. In 90 experiments with the ATRON self-reconfigurable robot we demonstrated success in between 40-94% of the docking attempts depending on approach angle and that a complete docking sequence takes 16 seconds. While this a modest step towards making fast and reliable docking, we conclude that behaviors that exploit morphology have a high potential and are a good complement to sensor-only approaches to docking. Acknowledgements. This work was supported by The Danish Agency for Science, Technology and Innovation project no. 274-06-0413.

References 1. Bererton, C., Khosla, P.: Toward a team of robots with repair capabilities: a visual docking system. In: Proc. ISER Seventh International Symposium on Experimental Robotics, Berlin, Germany, pp. 333–342 (2000) 2. Castano, A., Shen, W.-M., Will, P.: Conro: Towards deployable robots with inter-robot metamorphic capabilities. Autonomous Robots 8(3), 309–324 (2000) 3. Fukuda, T., Nakagawa, S.: Dynamically reconfigurable robotic system. In: Proc. IEEE Int. Conf. on Robotics & Automation, vol. 3, pp. 1581–1586 (1988) 4. Groß, R., Bonani, M., Mondada, F., Dorigo, M.: Autonomous self-assembly in a swarm-bot. In: Murase, K., Sekiyama, K., Kubota, N., Naniwa, T., Sitte, J. (eds.) Proc. 3rd Int. Symp. on Autonomous Minirobots for Research and Edutainment, Berlin, Germany, pp. 314–322. Springer, Heidelberg (2005) 5. Groß, R., Bonani, M., Mondada, F., Dorigo, M.: Autonomous self-assembly in swarm-bots. IEEE Transactions on Robotics 22(6), 1115–1130 (2006) 6. Murata, S., Kurokawa, H., Kokaji, S.: Self-assembling machine. In: Proc., IEEE Int. Conf. on Robotics & Automation, San Diego, California, USA, pp. 441–448 (1994) 7. Murata, S., Yoshida, E., Kamimura, A., Kurokawa, H., Tomita, K., Kokaji, S.: M-TRAN: self-reconfigurable modular robotic system. IEEE/ASME Transactions on Mechatronics 7(4), 432–441 (2002) 8. Murata, S., Kakomura, K., Kurokawa, H.: Docking experiments of a modular robot by visual feedback. In: Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, Beijing, China, pp. 625–630 (2006) 9. Østergaard, E.H., Kassow, K., Beck, R., Lund, H.H.: Design of the atron latticebased self-reconfigurable robot. Autonomous Robots 21(2), 165–183 (2006) 10. Payne, K., Everist, J., Hou, F., Shen, W.-M.: Single-sensor probabilistic localization on the seres self-reconfigurable robot. In: Proceedings, 9th International Conference on Intelligent and Autonomous Systems, Tokyo, Japan, pp. 207–216 (2006)

452

K. Stoy et al.

11. Raper, K.B.: Pseudoplasmodium formation and organization in dictyostelium discoideum. Journal of the Elisha Mitchell Scientific Society 56, 241–282 (1940) 12. Roufas, K., Zhang, Y., Duff, D., Yim, M.: Six degree of freedom sensing for docking using ir red emitters and receivers. In: Rus, D., Singh, S. (eds.) Experimental Robotics VII. Lecture Notes in Control and Information Sciences, vol. 271, pp. 91–100. Springer, Heidelberg (2001) 13. Rubenstein, M., Payne, K., Will, P., Shen, W.-M.: Docking among independent and autonomous conro self-reconfigurable robots. In: Proceedings, IEEE International Conference on Robotics and Automation, New Orleans, USA, pp. 2877–2882 (2004) 14. Rus, D., Vona, M.: A physical implementation of the crystalline robot. In: Proc. IEEE Int. Conf. on Robotics & Automation, San Francisco, USA, pp. 1726–1733 (2000) 15. Shen, W.-M., Will, P.: Docking in self-reconfigurable robots. In: Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, Maui, Hawaii, USA, vol. 2, pp. 1049–1054 (2001) 16. Yim, M., Duff, D.G., Roufas, K.D.: Polybot: A modular reconfigurable robot. In: Proc. IEEE Int. Conf. on Robotics & Automation, San Francisco, CA, USA, pp. 514–520 (2000) 17. Yim, M., Shirmohammadi, B., Sastra, J., Park, M., Dugan, M., Taylor, C.J.: Towards robotic self-reassembly after explosion. In: Proceedings IEEE/RSJ International Conference on Intelligent Robots and Systems, San Diego, CA, USA, pp. 2767–2772 (2007) 18. Yim, M., Zhang, Y., Roufas, K., Duff, D., Eldershaw, C.: Connecting and disconnecting for chain self-reconfiguration with polybot. IEEE/ASME Transactions on Mechatronics 7(4), 442–451 (2002) 19. Zhang, Y., Roufas, K., Eldershaw, C., Yim, M., Duff, D.: Sensor computation in modular self-reconfigurable robots. In: Siciliano, B., Dario, P. (eds.) Proc. Experimental Robotics VIII. Springer Tracts in Advanced Robotics, vol. 5. Springer, Heidelberg (2003) 20. Zykov, V., Mytilinaios, S., Desnoyer, M., Lipson, H.: Evolved and designed selfreproducing modular robotics. IEEE Transactions on Robotics 23(2), 308–319 (2007)

Reconfigurable Modular Universal Unit (MUU) for Mobile Robots Shugen Ma, Changlong Ye, Bin Li, and Yuechao Wang

Abstract.1 A modular universal unit (MUU) implements Pitching, Yawing and Rolling DOF is presented in this paper. By connecting these MUUs in different forms, we can develop the 3D snake-like robot and other mobile robots. A series of passive rollers around the cylindrical aluminum shell of MUU form an actuating wheel for the omnidirectional mobile robot. The communication and replacement is feasible due to high integration of MUU. Two or more MUUs can be connected to make up mobile robots to move, which exhibits high mobility and agility of MUU. Some typical configurations of robot composed of these MUUs are presented. The kinematics and controllability of proposed robots are analyzed for self-reconfigurable commission. Some locomotion experiments are given to show characteristic of MUU.

1 Introduction A self-reconfigurable robot usually consists of a lot of interconnected heterogeneous or homogenous robot module[1-5]. The modules are highly integrated for control, sensing, communication and connection. Such robots can change their configuration and move from one place to another place through reassembling some modules’ position. Self-reconfigurable robots have various advantages compared with the conventional robotic system. Due to the adaptation and self-repair ability, Shugen Ma, Changlong Ye, Bin Li, and Yuechao Wang State Key Laboratory on Robotics, Shenyang Institute of Automation, CAS, Nanta Street 114, 110016, Shenyang, China e-mail: [email protected], [email protected], [email protected] Shugen Ma Department of Robotics, College of Science and Engineering, Ritsumeikan University, Shiga-ken 525-8577, Japan Changlong Ye Shenyang Institute of Aeronautical Engineering, Daoyi Street 37, Shenyang, 110136, China Shugen Ma and Changlong Ye Harbin Institute of Technology, Xida Street 92, Harbin, 150001, China

454

S. Ma et al.

this class of robot shows more potential applications in the unstructured and hazardous environment such as the nuclear power station and outer space. Many researchers have developed the self-reconfigurable robotic system. The ATRON is latticed-based and able to self-reconfigure in 3D, which has a limited mobility in structure for the motion constraints[1]. Also the hardware prototype of modules are able to self-reconfigure include MTRAN[2], 3D-unit[3], Molecule[4] and I-cubes[5]. The related motion control and planning work are studied for the efficient reconfiguration and locomotion[4]. In this paper, a modular joint mechanism implementing three DOFs is developed for reconfigurable robots. It shows favorable performance on movement and reconfiguration, because two or more units can be connected to move or maneuver effectively under control. This unit with a series of free rollers around its cylindrical shell can also be used as the wheel of mobile robot, which expands the potential applications of MUU. As shown in Fig.1, the snake-like robot, two units(car-like vehicle), four driven omnidirectional vehicle and manipulator are grouped in team. These reconfigurable robots composed of MUU exhibit broad application of this module. Fig. 1 Some robot composed of MUUs

2 Mechanism Design of MUU Mechanism design of modular unit is the basis of reconfigurable robot. Many modular units with only one DOF have been developed for the simple structure. However, only a group of units connected together can realize movement or reconfiguration because this class of units is limited by the mechanism and lake of agility. Therefore, we develop a modular unit with three DOFs named as MUU to increase its mobility.

2.1 Structure of MUU The joint unit MUU(Modular Universal Unit) is developed with Pitch, Yaw and Roll DOF. Three FutabaS3305 servomotors with high torque output have been selected as joint actuators in this unit. Both Pitch and Yaw motions are driven in coupling through differential mechanism, as shown in Fig.2(a). The differential mechanism composed of three bevel gears adequately utilizes the system actuation and increases the ratio of actuation to mass. The reduction ratio 2:1 is setup between FutabaS3305 servomotor and bevel gear. This MUU with differential mechanism can lift two modular units. Potentiometers are fixed on the end of each

Reconfigurable Modular Universal Unit (MUU) for Mobile Robots

455

Table 2.1 Specifications of MUU DOF

3(pitch 1,yaw 1,roll 1)

Motor

FutabaS3305 3

Dimension[mm ]

172x120(circumferential)

Workspace of joint[°]

-90~90 (pitch and yaw) Continuous rotation (roll)

Mass of joint[kg]

1

Torque

3.56Nm(pitch, yaw) 1.78Nm(roll)

Angular velocity of joint[°/sec]

150 (pitch,yaw,roll)

motion axis to detect angle information for control, as shown in Fig.2(a). Roll motion is generated by another servomotor through 2:1 reduction radio. Both Pitch DOF and Yaw DOF can swing from -90° to 90° position. All three axes intersect at one point to avoid the singularity of motion shown in Fig.2(b). Here, α , β , γ are variables of three axes, respectively. Roll DOF

Potentiometer

Reduction Gears

DC Motor

Reduction Gears

A

A Direction

A Battery

Pitch Axis

Differential Gear Bevel 1 Bevel 2 Bevel 3 Yaw Axis

a. Inner structure of MUU

β

Control module

γ

α

Pitch Axis Roll Axis Yaw Axis Pitch Axis Roll Axis Yaw Axis b. Arrangement of 3DOF in the unit

Fig. 2 Construction of MUU

c. Hardware of MUU

456

S. Ma et al.

A Panasonic lithium battery in the unit supplies the power to these three motors. Also the control module carrying a 16Bit Fujitsu processor manages its own control information. CAN bus is chosen to communicate among all units. For convenient connection, radio control will be realized in future work. Outer shell of this joint unit is an aluminum cylinder, on which a series of passive rollers are fixed to obtain the frictional property for the creeping movement, as shown in Fig.3. This unit thus is easy to slip along the trunk, and difficult to slip in normal direction orthogonal to the trunk. The cylindrical shell can be used as a big drive wheel to compensate the lack of propulsion due to passive rollers. The specifications of MUU are listed in table 2.1. Its outer dimension is 172mm(long)x 120mm(circumferential) and weight is about 1kg. The hard components are shown in Fig.2(c).

2.2 Body Wheel The body wheel is designed to realize the contact with ground, which provides robot movement with propulsion. On one hand the unit or robot need to contact the ground to get propulsion to move in one direction, i.e. one directional movement should be constricted. On the other hand it should avoid friction to slide in the other direction to move. Therefore, there are free rollers installed around the body to rotate freely along unit body. These rollers also form body wheel around unit to realize the contact for propulsion. The rollers make it easy to slide along the unit body and difficult to slide in the direction normal to unit. The spindle shape of roller is derived from the shape of body wheel, whose radius is equal to the radius of spindle shape in vertical section. The rollers are fixed on the middle of the line from the pitch axis to the next pitch axis of connected MUU, as shown in Fig.3. This location of rollers benefits the serpentine locomotion and robot modeling, mainly comes from the research in [6] for the snake-like robot movement. The force acts on the body in the normal direction, while the body wheel rotates, and the other forces in other directions are released by the free rollers, as shown in Fig.4. From view of this point, the body wheel is an omni-directional wheel, which benefits the locomotion of the robot. The body wheel rotation not only can be actuated by the roll DOF, but also can be actuated by the combination of Pitch and Yaw DOFs. This is an important characteristic of MUU for the locomotion of reconfigurable robot. When the pitch DOF is actuated to 90 degree position as shown in Fig.4, the yaw axis coincides with the axis of body wheel. Consequently, after the L-shape edge is fixed, the body wheel will be actuated to rotate around yaw axis by driving the Yaw DOF. At this configuration, the MUU can be used as a drive wheel of vehicle, which increases the motion performance of reconfigurable robot. In this paper, we name this configuration of MUU as wheel configuration.

2.3 Decoupling the Input Variable Each MUU implements three degrees of freedom, and among which the Pitch DOF and Yaw DOF are driven in coupling while the roll DOF action leads

Reconfigurable Modular Universal Unit (MUU) for Mobile Robots

457

another rotation. The decoupling of three inputs must be resolved firstly to realize separate control of all three DOFs. The decoupling kinematics is given as follows,

⎡α ⎤ ⎡1 −1 0 ⎤ ⎡ϑ1 ⎤ ⎢ β ⎥ = 1 ⎢1 1 0 ⎥ ⎢ϑ ⎥ ⎢ ⎥ 4⎢ ⎥⎢ 2⎥ ⎢⎣ γ ⎥⎦ ⎢⎣0 0 2 ⎥⎦ ⎢⎣ϑ3 ⎥⎦

(1)

where α , β and γ are the angles to define the unit posture, as shown in Fig.2b. ϑ1 , ϑ2 and ϑ3 are the output angles of DC motor, respectively. The reduction ratio 2:1 has been considered in this equation. Cone path

Next Pitch Axis

Free slide Roller Position

Propulsion

Fixed edge

Pitch Axis

Yaw axis

Fig. 3 Passive rollers and its location

Fig. 4 Body wheel configuration

3 Some Typical Configurations The developed MUU is mainly for a 3D snake-like robot and other mobile robots. As a modular unit, it can form some other desired configurations. If we add the connection mechanism and some sensors on the MUU interface, it can connect or disconnect to another unit automatically to complete self-repair or self-reconfiguration, which will be researched in future work. At present, they are connected by screw. Firstly, we develop a 3D snake-like robot by connecting the MUU in series, as shown in Fig.5. This snake-like robot with high propulsion and maneuverability can conquer some obstacles. Secondly, the vehicle with two units can move like a car, which can be controlled to move from point to point. In this case, one MUU is used as drive wheel and the other MUU is used as a caster. Two MUUs have three contact points to get the stable pose. From Fig.6, we can see that both the passive roller and the

Three contact points

Fig. 5 The movement of one MUU

Fig. 6 The movement of two MUUs

458

S. Ma et al.

cylinder edge of rolling MUU are contacting the ground, which prevents the slide along the body. Otherwise, two MUUs connection can be free around a point. From Fig.7, we can get the kinematics model of two MUUs connection vehicle. Geometric constraints: x2 = x1 + l1C (θ ) + l2 C (θ + β 2 )

(2)

y2 = y1 + l1 S (θ ) + l2 S (θ + β 2 ) Velocity constraints: x&1 S (θ ) − y&1C (θ ) = 0 x&2 S (θ + β 2 ) − y& 2 C (θ + β 2 ) = 0

(3)

Therefore, the locomotion control equation can be written as following,

0 ⎡ C (θ ) ⎤ ⎡ ⎤ ⎡ x&1 ⎤ ⎢ ⎥ ⎢ ⎥ θ ( ) 0 S ⎢ y& ⎥ ⎢ ⎥ ⎢ ⎥ ⎢ 1 ⎥ = ⎢ S ( β ) ⎥ R β& + ⎢ ⎥ β&2 l2 1 2 & ⎢θ ⎥ ⎢ ⎥ ⎢− ⎥ ⎢ & ⎥ ⎢ l1C ( β 2 ) + l2 ⎥ ⎢ l1C ( β 2 ) + l2 ⎥ ⎣β2 ⎦ ⎢ ⎥ ⎢ ⎥⎦ 0 1 ⎣ ⎦ ⎣

(4)

y2

β2 x2

2

l1

l2

p Contact edge

y1

θ x1

y Contact edge

o

x

P

y

1

2

1

o

x

Fig. 7 Kinematics model of two MUUs Fig. 8 Two MUUs in parallel configuration

where ( xi , yi ) are the coordinates of a reference point on the MUU, R is the radius of this kind of mobile robot, subscript i means the number of unit, θ is its orientation, respectively. C (•) := cos(•) , S (•) := sin(•) , l1 , l2 express the distance between rotational axis of the drive wheel and the joint of caster, the distance between the joint and the passive roller of caster, respectively. From equation (4), we can find that the two MUUs’ vehicle is controllable. Though the MUU has large workspace to minimize the turn radius, singularity of the two MUUs connection would appear at the condition of minimum turn radius. The orientation of movement is not under control anymore when the Yaw angle of the second unit is set at the 90 degree, because two directions of the

Reconfigurable Modular Universal Unit (MUU) for Mobile Robots

459

caster and the drive is perpendicular. When two MUUs are at the singularity of turn motion, the connection vehicle can shift it locomotion method from one drive wheel and one caster to two drive wheels, as shown in Fig.8. There are still three contact points including passive rollers and one contact edge to support the vehicle on the ground, which prevent slip along the body. When two MUUs drive in same direction with same speed, the vehicle would move forward. When two MUUs drive in opposite direction with same speed, the vehicle would move around the geometrical center P. Finally, the developed MUU can be used as the driving wheels of a four-wheel-driven vehicle, which is an omni-directional vehicle. As shown in Fig. 9, the vehicle with four supporting wheels can be driven by four wheels to perform omni-directional motions. As shown in Fig.10, the geometric constraint of MUU 1 is given by, x1 = xv + l1C (θ v ) + l2 S (θ v ) y1 = yv + l1 S (θ v ) − l2 C (θ v )



(5)

and its velocity constraint is given by,

x&1C (θ1 ) + y&1 S (θ1 ) = R β&1 

(6)

Similarly, we can obtain the constraint function of other MUUs. The function of the four-wheel-driven vehicle is thus given by, ⎡ C (θ ) ⎢C (θ ) ⎢ ⎢ C (θ ) ⎢⎣C (θ ) 1

S (θ 1 )

⎤ x& ⎡ )⎥ ⎥ ⎢ y& )⎥ ⎢ & ⎥ ⎣θ )⎦

a1 C (θ1 ) + b1 S (θ1 )

2

S (θ 2 )

a 2 C (θ 2 ) + b2 S (θ 2

3

S (θ 3 )

a 3 C (θ 3 ) + b3 S (θ 3

4

S (θ 4 )

a 4 C (θ 4 ) + b4 S (θ 4

v

v

v

⎡ β& ⎤ ⎢ β& ⎥ = R⎢ ⎥ ⎢ β& ⎦ ⎢& ⎣β

1

2

3

4

⎤ ⎥ ⎥  ⎥ ⎥⎦

(7)

where ai , bi are the wheel position parameters that are related to parameters l1 , l2 , θ , β& are the angle parameters to the vehicle body and the angular velocity of the i

i

wheels, respectively. Subscript i describes the wheel number. ( xv , yv , θv ) are the vehicle body position and posture, respectively. R is the radius of body wheel. From equation (7), we can see that this vehicle can perform omni-directional motion and can move along x or y axis and around its center. 2

yv

l2

3

y

l1 θv

xv

y&1

θ1 1

x&1

4

o

Fig. 9 A vehicle with four MUUs

x

Fig. 10 Movement of the vehicle with four MUUs

460

S. Ma et al.

4 Experiments As basic unit of reconfigurable robot, Two MMUs’ connection vehicle can move to any position like a car or in parallel configuration, as shown in Fig.11. The Fig.12 shows that four-wheel omnidirectional mobile robot do turning and forward movements.

Fig. 11 Two MUUs movement

Fig. 12 Turning motion and forward motion of a four-wheels-driven vehicle

5 Conclusion and Future Work In this paper, we describe a novel unit named MUU for the reconfigurable robot. This MUU implements three DOF, among which Pitch and Yaw DOF are driven by couple drive with differential mechanism. There are many configurations that MUUs can form such as the snake-like robot, car-like robot. The kinematics of some reconfigurable robot consisting of MUUs is given for the robot control. Kinematic analysis of MUU and reconfigurable robot show that the MUU is developed with favorable performance. Experimental results validate the mobility and agility of MUU. In future work, the connection interface and control system will be studied for the self-reconfiguration. Acknowledgments. This work was partially supported by 863 projects (No2007AA041703 and No.2006AA04Z254).

References [1] Johan Christensen, D.: Evolution of shape-changing and self-repairing control for the ATRON self-reconfigurable robot. In: Proceedings of the 2006 ICRA, Orlando, Florida, pp. 2539–2545 (May 2006) [2] Murata, S., Yoshida, E., Kamimura, A., Kurokawa, H., Tomita, K., Kokaji, S.: M-TRAN: self-reconfigurable modularrobotic system. IEEE/ASME Transactions on Mechatronics 7(4), 431–441 (2002) [3] Murata, S., Kurokawa, H., Yoshida, E., Tomita, K., Kokaji, S.: A 3-d self-reconfigurable structure. In: Proceedings of 1998 ICRA, pp. 432–439 (1998)

Reconfigurable Modular Universal Unit (MUU) for Mobile Robots

461

[4] Kotay, K., Rus, D.: Algorithms for self-reconfiguring molecule motion planning. In: Proceedings of 2000 IROS, pp. 823–828 (2000) [5] Unsal, C., Kiliccote, H., Khosla, P.: I-cube: a modular self-reconfigurable bipartite robotic system. In: Proceedings, SPIE Sensor Fusion and Decentralized Control in Robotic Systems II, vol. 3839, pp. 258–269 (1999) [6] Ma, S.: Development of a Creeping Snake-Robot. Int: Journal of Robotics and Automation 17(4), 146–153 (2002)

Part VII

Multi-robot Systems

Design and Analysis for AGV Systems Using Competitive Co-evolution Ryosuke Chiba, Tamio Arai, and Jun Ota

1 Introduction Recently, material-handling systems with Automated Guided Vehicles (AGVs) are being used in manufacturing factories. These programmable AGVs circulate on a guide-path and transport materials in factories (see Fig.1). Generally, one of the important design problems associated with the development of AGV systems is a flow-path network design problem [1]. The flow-path network design problem is how to design the placement of guides on the factory floor. The network restricts the area where AGVs run, but it is necessary for AGV systems to improve the precision of AGV movements. This paper describes the solutions to the flow-path network design problem. The AGV systems are usually evaluated with following items. • Effectiveness : AGVs should transport demanded items which are needed for products in a day/week/month with a certain fleet size. • Robustness : At any day/week/month, AGVs should transport demanded items which will change the contents at every day. • Simplicity : In AGVs systems, their movements should be easily understood by workers because of their safety. In researches of this field, the design problem is partially solved. The flow-path network is either uni-directional or bi-directional in AGV systems. The uni-directional flow-path network may limit efficiency and productivity [2]. In the Ryosuke Chiba Tokyo Metropolitan University e-mail: [email protected] Tamio Arai and Jun Ota The University of Tokyo e-mail: {arai-tamio,ota}@robot.t.u-tokyo.ac.jp

466

R. Chiba et al.

central control system

camera storage

belt-conveyer

AGV guidance

workshop, machine Fig. 1 AGV system

case of the uni-directional AGV systems with fixed aisles, the problem of determining the path directions is formulated by Gaskins and Tanchoco [3]. Kaspi and Tanchoco [4] considered a set of possible flow path arcs and modeled the problem of selecting path directions. Gaskins and Tanchoco [5] proposed a bi-directional flow using of multiple parallel lanes with upper-capacity limits. In their model, they however designed the flow-path network as uni-directional only and assumed that AGVs would not have collisions with other AGVs on the path. Therefore, bi-directional flow-path network cannot be modeled in the same way. Bi-directional flow-path network is more efficient than uni-directional one. However, AGVs often cause deadlocks with just a simple routings in bi-directional flow-path networks and uni-directional flow-path network may avoid the deadlocks. Therefore, a uni and bi directioncal flow-path is combined for the effective and nondeadlocked system in this paper. The design problem of bi-directional network is solved in our previous research [6]. However, the designed flow-path network is optimized against a certain task. From our previous research, the AGV system with the optimized network against a certain task is not effective against other tasks. All above researchers also optimize against a certain task. In this design process, when the task changes to another task, the flow-path network should be designed again from the beginning. The flexibility is one of the strong merits in AGV system. Therefore, the robust AGV system is the more efficient transportation system than other systems. In this paper, the method to generate the robust flow-path network design with Competitive Co-evolution is proposed. Both the network and the task evolve competitively as in Fig.2, and finally, a robust network and difficult tasks are designed. The robust network means that the minimum number of conveyance is large with the network to some tasks. Namely, the mini-max problem should be solved.

Design and Analysis for AGV Systems Using Competitive Co-evolution

467

Network as input

Inputs Environment

Number of AGVs

Network Design Process

Task Design Process

Routing program

Task as input

Fig. 2 Robust flow-path network design process

To realize the method, three issues should be solved. 1. Uni/bi directional flow-path network should be designed efficiently. 2. Difficult tasks should be designed against AGV systems. For robust system design, one task is not enough as a test-set to design and evaluate a robust system. Therefore, an effective task-set should be designed. 3. A ”solution rotation problem” in the competitive co-evolution should be solved for the robust network against tasks. The details of the ”solution rotation problem” are mentioned in section 4. The first issue is resolved with dividing environment with square cells and calculating connections between cells using Genetic Algorithm. The second issue is also solved with GA because the number of possible tasks is very large. These first and second issues will be mentioned in section 3. The design of a task-set is achieved by ”packaging” mentioned in section 4.The third issue is done with packaging method in the competitive co-evolution. This will be mentioned in section 4. The method has been proposed in our previous work [7] partly. Our aim of this paper is to verify the proposed method with some kinds of number of AGVs and to analyze the method. In section 5, the proposed method is applied to a design simulation of AGV systems. And this paper is concluded in section 6.

2 Model of AGV Systems 2.1 Settlement The assumptions and system requirements are described in this section. Environment refers to the factory layout followed by the AGVs, which includes the pick-up points where the supplies are stored items and the drop-off points where the machines to wait for the items are located. Similarly, task refers to the conveyance requirements. The conveyance requirement includes pick-up point number, drop-off point number, and the time when the requirement occurs. An example of the environment for a material handling system is shown in Fig.3. The shaded areas in this figure indicate environmental obstacles. P and D in this figure indicate pick-up point (storage) and drop-off point (machine) respectively.

468

R. Chiba et al.

Fig. 3 Work environment

D

P

D

P

D

P

D

P

This environment is illustrated on a two-dimensional plane that can be divided into square cells. The AGVs can move to up, down, right, and left cell with one step. The AGVs operate throughout the geometrical model of the factory floor. This environment is based on a real car factory’s environment. As above described, the task is a group of conveyance requirements. The conveyance requirements occur at every fixed period. When an AGV is assigned the requirement of the task to, the AGV acts on the orders to convey items from the pick-up and drop-off points. The transporter routing as a input is reactive approach [8], where all of AGVs takes the shortest distance path to their destinations ignoring the other AGVs. This approach is very simple and often applied to a real system. However, it is easy to deadlock. Therefore, it is very important and difficult to design a network to be non-deadlock on this routing.

2.2 Problem Definition The AGV systems must be designed with the considerations as follows. • Task should be completed within a certain period. • The network path length should be kept to a minimum. The systems are evaluated in this order, so completion of task is the top priority. The simple (non-crowded) flow-path netowork is better for the workers if the systems have fewer aisles of AGVs and small network because the workers share the floor with them. As mentioned later, the fitness function of systems is defined according to these requests.

3 Design Processes 3.1 Flow-Path Design The flow-path network should be designed with adjusting to routings and a number of AGVs. To be mentioned in section 1, uni/bi directional flow-path network design problem should be solved for an effective AGV system. A uni/bi directional flow path system is, however, very complex to design. In [9], the effectiveness and

Design and Analysis for AGV Systems Using Competitive Co-evolution 0 0 1 1 1 0 1 0 1 0 0 0 1 0 0 0 1 0 0 0

0 0 0 0 0 1 0 1 0 1 0 0 0 0 0 1 0 0 0 0 1 0 1 0 1 0 1 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 1 1 1 1 1 1 1 0 0 0 0 0

469

decode

Fig. 4 Decoding genes into flow-path network

complexity is researched and the flow-path netowrk is just designed with their heuristics approach. Therefore, the following methods are conceivable to construct a bi-directional flow-path network. • Scheduling all movements of AGVs. • Using a graph flow model. • Solving with simulation-based optimization using a Genetic Algorithm. It takes too much time to calculate using a method that requires the scheduling of all movements of the AGVs. A method that use graph flow model cannot be used to construct a bi-directional flow path. Therefore, the simulation-based optimization is applied to this problem using Genetic Algorithm in this paper. Flow-path network is designed by searching for useless paths by means of simulation. When a path is useless, the efficiency of the conveyance does not change, even when the AGVs don’t use the path. If all flow paths can be used in an environment, the AGVs select many useless paths. This problem is however non-liner and difficult. Therefore, a genetic algorithm is used for the flow path design for this reason. In our method, the genes represented binary indicate one-way connections of the cells (see Fig.4). They restrict the one-way connection with the up, down, left and right cells. For example, if a gene indicating the connection with the upper cell is zero, AGVs cannot move to the upper cell. To overlap one-way connections between two cells indicate a both-way connection which is a bi-directional path. By the connections, an individual represents a flow path. The number of the genes of an individual is about twice as many as that of the cells. Those individuals are tested through simulations. The one of the reasons using GA is the number of possible networks are very large. In this representation for networks, the length of gene is about 4 times of cell size. For example, on Fig.3, the number of cells is 60, so the gene length is 240. In this case, AGV system has 2240 possible networks. Genetic Algorithm is strong to such a large-scale optimization.

3.2 Task In the environment at Fig.3, the number of pick-up points is four and that of drop-off points is also four. Thus, the number of possible transport requirements is 42 = 16.

470

R. Chiba et al.

I Generation t-1

Networks

Tasks

IV

Generation t

Networks IV

Generation t+1

Networks

III

Evolve

I

Tasks

III I

IV

II

Procedure

II

Tasks III

II

Fig. 5 Evolving step in co-evolution; I:for a task evaluation, an individual are chosen from the network group. II:The task group evolves according the procedure of GA. III:for a network evaluation, an individual is chosen from the task group. IV:The network group evolves according the procedure of GA.

A task in this paper is a set of the requirements. For instance, when a task is consisted of 50 requirements, the number of possible tasks is 1650 = 2200 . This number is such large that a flow-path network cannot be evaluated against all of tasks. With this reason, GA is applied to design a difficult task. The length of the gene is 4 times of the number of transport requirements. In previous instance, the length is 200. As mentioned in previous subsection, such a kind of problem can be solved with GA.

4 Competitive Co-evolution 4.1 Methodology An effective flow-path network and difficult tasks should be designed simultaneously, because the evaluations of these two elements depend on the other element. To solve this problem, competitive co-evolution is adopted. In this method, at beginning, some individuals for flow-path networks and another some individuals for tasks are prepared. And then, the group of networks and the group of tasks evolve by turn (see Fig.5). In this process, when an individual is evaluated, the individual chose an individual from among the other group as the competitor. This system is described in detail by Sims1994 [10]. The whole algorithm is shown in below. 1. Set the initial groups for a network and a task. In these groups, some individuals are included respectively. And set 0 to the number of generation. 2. Evaluate the individuals representing networks. In this term, select a competitor from task group.

Design and Analysis for AGV Systems Using Competitive Co-evolution

System Group

Individuals

against

471

Task Group Package

Fig. 6 Packaging

3. When all of individuals are evaluated, evaluate the individuals for tasks. 4. Evolve the group of networks and evaluated them. 5. Evolve the group of tasks and evaluate them. Increase 1 to the number of generation. 6. If the number of generation is a certain end number, finish this calculation. Otherwise, go back to 4.

4.2 Packaging Generally, each individual is tested against the best competitor on the competitive co-evolution [10] [11]. This strategy is called ”best vs. all”. However, with this competitive co-evolution, an effective flow-path network and difficult tasks cannot be designed because of ”solution rotation problem”. The example of this phenomenon is shown in below. In this competitive co-evolution system, when a network na are generated against a certain task ta , tasks try to beat na and a difficult task tb is generated. Then, to be effective against the task tb , a network nb are obtained. If the hard task against nb is ta , the networks however evolve to na again. In this case, these solutions are generated in rotation repeatedly. To solve this problem, a strategy named ”packaging” is proposed. In this method, when a difficult task tk are found, the individual are added into a package P as pi . And when a individual of a network are evaluated against all of the packaged tasks (∀p). With this method, a robust network against various tasks can be generated. Individuals for tasks are properly added to the package according to the rule that the best-performing individual is added after the tasks are evaluated. If the best have already packaged, no new individual is packaged. The evaluation is done against the best network in previous generation. This packaging method makes it possible to generate a robust and effective flowpath network and a group of difficult tasks without the ”solution rotation problem”.

472

R. Chiba et al.

4.3 Fitness Function The fitness of a flow-path network is calculated with all of the tasks in package and the lowest value is the fitness of the individual for the network. For the evaluations of individuals, the fitness function of GA is designed as follows. f itness(ni ) = min{∀p K1 × x1(ni , p) + (1.0 − K1) × x2 (ni )}

(1)

where K1 is a constant value and setted 0.8 in this paper. The values in this formula are as follows. • x1 : the number of requirements that AGVs achieve. The formula of the value is given below.

x1 =

number o f achieved conveyances number o f requirements

(2)

• x2 : the degree of the flow-path length. The shorter one is the higher value. The formula of the value is given below.

x2 =

number o f genes which value is 0 number o f genes o f network

(3)

The fitness function indicates that the individuals as follows are easy to survive. • The system that can make AGVs convey in quantity. • The path network which is not crowded. The suitable system mentioned in section 2 can be constructed by this fitness function. At the evolutions of tasks, the fitness is calculated as follows. f itness = 1.0 − {K1 × x1 + (1.0 − K1) × x2 }

(4)

This function indicates that the hard task can be easy to survive.

5 Experiment and Comparison 5.1 Settlement To verify the proposed method, some experiments through simulations are carried out. There are 3 kinds of settlements that must be described. Those are, i)about Genetic Algorithm, ii)about AGV system and iii)about co-evolutional method to be proposed. The details of those are as follows.

Design and Analysis for AGV Systems Using Competitive Co-evolution D

P

D

P

D

P

D

P

(a) Flow-path network with 2 AGVs

D

P

D

P

D

P

D

P

(b) Flow-path network with 3 AGVs

473

D

P

D

P

D

P

D

P

(c) Flow-path network with 4 AGVs

Fig. 7 Result of flow-path networks; blue arrows indicate one-way path and red arrows indicate both-way path

1. The operations in GA: • The number of individuals for a flow-path network is 128 and that for a task is 256. • The number of generations is 4500 including both networks (1500 generations) and tasks (3000 generations). • Cross over: one point, Possibility of cross over: 0.7, Possibility of mutation: 0.01, Roulette and elite selection strategy. 2. Inputs in AGV systems • The environment of AGV system is shown in Fig.3. In this environment, the number of connections between the cells is 208. That is, the length of the genes to represent a flow-path network is 208. • All of tasks consists of 48 conveyance requirements. Therefore, the number of possible tasks is 1648 = 2192 and the length of the genes to represent a task is 192. • The number of AGVs is 4 and their initial positions are fixed. They can convey only one item at once. • AGVs can move to 4 directions (up, down, left and right) within a certain time. This time is defined as one step. • The transporter routing is reactive approach as mentioned in section 2. • The simulation is over at 1200/(number o f AGV s). 3. Operation in co-evolution • Networks evolve 5 times and then tasks do 10 times, and this cycle is repeated. To make each group evolve by a generation is not effective, because each group does not evolve enough in only one generation.

5.2 Results of Experiments The results of the above-mentioned simulation are shown in this subsection. At the end of these simulations, flow-path networks are shown in Fig.7 is generated. These network look not to be crowded. The transition of system effectiveness is shown in Fig.8. At the beginning, the value changes very much at every five or ten generations. This swing indicates that

474

R. Chiba et al. 1

0.9 0.8 0.7

Fitness

0.6 0.5 0.4 0.3 0.2 0.1 0 0

500

1000

1500

2000

2500

3000

3500

4000

4500

Generation

Fig. 8 Fitness transition with 4 AGVs

the package is not enough to make a robust flow-path network. As the network becomes robust, the tasks do not perform well and it becomes difficult to design a difficult task. The design time for 3 AGVs network is longer than the design times for the networks with the other number of AGVs. It tasks about 50 hours to calculate the network and the tasks with 3 AGVs using 16 CPUs (Pentium Xeon 2.2[GHz]).

5.3 Discussion From these results, Some insights can be obtained. First, the designed networks with our method are not crowded and makes it possible that AGVs can transport all of 48 items. It indicates very effective for AGV systems to apply GA to design a flow-path network. These two requests, simple network and effective transportation, are not easy to achieve because the relation between non-crowded and many conveyance is trade-off. Secondly, the feature of the designed networks are the combination of typical single loop structure [12]. This structure which is proposed to be based on heuristic approach is very famous AGV system and utilized in many real factories as a robust flow-path network. Therefore, our designed flow-path network is very appropriate for the robust AGV system because all of AGVs takes the same directions (see Fig.9). One of our contributions is our meta-heuristics approach can make the famous network structure based on the heuristics approach. Thirdly, with the competitive co-evolution, both a robust network and an effective task-set can be generated at the same time in any number of AGVs. From Fig.8, the system values swing between low value and high value at the beginning, but, after enough evolution, the values converge at a certain value. This is the evidence that the designed flow-path network becomes robust and the package includes enough tasks. Finally, the network with 3AGVs is more complex than that with 4 AGVs. This fact indicates that the convergence will be early in the difficult condition because the

Design and Analysis for AGV Systems Using Competitive Co-evolution D

P

D

P

D

P

D

P

(a) Routing rule with 2 AGVs

D

P

D

P

D

P

D

P

(b) Routing rule with 3 AGVs

475

D

P

D

P

D

P

D

P

(c) Routing rule with 4 AGVs

Fig. 9 Routing rules of AGVs; the arrow at center indicates a view of AGVs movement

difficult tasks can be found early in this case. At the same time, in the easy condition (with 2 AGVs), the convergence is also early and the simple flow-path network can be designed, since the difficult task is a few. Therefore, it can be found that this problem is not so much simple with the proportional relation between the difficulty of the condition and the simplicity of the flow-path network. From the opinion that AGV system should be robust, our method makes it possible because of the result shown above. The method is useful for the design of a robust system.

6 Conclusion The design method for an effective and robust flow-path network is proposed in this paper for an AGV system. First, a flow-path network which is combined uni/bi directioal pathes can be designed not to be crowded with jast a simple transporter routings using Genetic Algorithm and the result shows in section 5. Secondly, the effective task-set for a robust AGV system can be design with GA and a packaging strategy and the results show in section 5. Finally, both a robust network and a difficult task-set can be simultaneously obtained using Competitive Co-evolution. Only by the design method at the same time, a robust flow-path network can be designed because networks depend on tasks and tasks depend on network. To evolve both with their interactions make it possible to generate the effective them. The designed flow-paths are analyzed based on the number fo AGVs which indicates the difficulty of the conditions. Acknowledgements. This work has been partially supported by a Grant-in-Aid for Young Scientists (B) from the Japanese Ministry of Education, Culture, Sports, Science and Technology.

References 1. Riopel, D., Langevin, A., Savard, G.: G-98-32, Les Cahiers du GERAD (1998) 2. Egbelu, P.J., Tanchoco, J.M.A.: International Journal of Production Research 24(5), 1075–1097 (1986)

476

R. Chiba et al.

3. Gaskins, R.J., Tanchoco, J.M.A.: International Journal of Production Research 25(5), 667–676 (1987) 4. Kaspi, M., Tanchoco, J.M.A.: International Journal of Production Research 28(6), 1023– 1030 (1990) 5. Gaskins, R.J., Tanchoco, J.M.A., Taghaboni, F.: International Journal of Production Research 27(1), 91–100 (1989) 6. Chiba, R., Ota, J., Arai, T.: Proc. of Int. Conf. Intell. Robots and Systems (IROS 2002), pp. 1820–1825 (2002) 7. Chiba, R., Ota, J., Arai, T.: Proc. of Int. Conf. Intell. Robots and Systems (IROS 2005), pp. 3164–3169 (2002) 8. Qiu, L., Hsu, W., Huang, S., Wang, H.: International Journal of Production Research 40, 745–760 (2002) 9. Rajotia, S., Shanker, K., Batra, J.L.: International Journal of Production Research 36(7), 1779 (1998) 10. Sims, K.: Artificial Life IV Proceedings, pp. 28–39. Brooks & Maes, MIT Press (1994) 11. Floreano, D., Nolfi, S.: Proceedings of the second International Conference on Genetic Programming, pp. 398–406 (1997) 12. Tanchoco, J.M.A., Sinriech, D.: International Journal of Production Research 30(3), 665–681 (1992)

Cooperative Control of Multi-robot Nonholonomic Systems with Dynamics Uncertainties and Control Time-Delays Junjie Zhang⋆ and Suhada Jayasuriya

Abstract. In practical applications, multi-robot systems may have to simultaneously deal with several tasks: collision-free maneuvers in dynamic environments; tracking certain desired trajectories; forming suitable patterns or geometrical shapes, and/or varying the pattern when necessary. These important issues are addressed in this paper. First a dynamic model for a robot prototype is presented. By appropriate coordinate transformation, this class of time-varying nonlinear systems, subject to modeling uncertainties and input time-delays, is further linearized into classical singularly perturbed systems. A robust compensation tracking controller is then developed and justified based on Lyapunov stability theorem. The compensated system effectively suppresses the perturbation effects to guarantee robust stability and tracking errors exponentially convergent to a bounded residual set. To achieve these typical complex formation missions, a previously developed two-layer hierarchical architecture for multi-robot nonholonomic systems is employed. In the upper layer, task specifications are assigned along with the desired behaviors in the task space and a null-space method is then utilized to integrate and prioritize these tasks and generate reference trajectories. In the lower layer, the local robust control scheme guarantees asymptotical tracking of the desired trajectories. A realistic case study illustrates the effectiveness of the modeling and robust control design.

Nomenclature OXY oxb yb

inertial coordinate system body-fixed coordinate system

Junjie Zhang and Suhada Jayasuriya Department of Mechanical Engineering, Texas A&M University, College Station, TX 77843 USA e-mail: {jjzhang,sjayasuriya}@tamu.edu ⋆

Address all correspondence to this author.

478

mi di l Ir Ip Id Xg /Yg ψ θi /θ˙i

J. Zhang and S. Jayasuriya

mass of the robot platform (i = r) and driving wheel (i = w) diameter of the circular robot platform (i = r) and driving wheel (i = w) distance between platform geometric center and center of mass moment of inertia of the robot platform about the zb -axis polar moment of inertia about the wheel axis diametral moment of inertia about diameter of the driving wheel X/Y coordinate of center of mass in frame OXY orientation angle of the platform in frame OXY angle/angular velocity of the right and left (i = R, L) driving wheels

1 Introduction Recent decades have witnessed a significant growth of research on cooperative control or formation control, which is principally motivated by the question of how best to prescribe desired global behaviors to an interconnected multi-agent system only using local interactions. Relevant work has been reported in a number of scientific fields, especially in biological science [1], physics [2], economics [3], as well as in robotics and control engineering [4, 5, 6]. As indicated in the future work of [7], feedback linearization techniques generally require accurate plant models to achieve exact linearization of closed-loop systems. However, there inevitably exist uncertainties in the constructed models for physical systems. Furthermore, in practice it is almost impossible to have noise and latency free data channels. Therefore, it is demanded that control design be robust enough to be capable of dealing with these uncertainty and time-delay issues. It is well-known that in addition to performance degradation, the manifestation of time-delays in a system can also cause extra disturbances and even deteriorate the stability of the closed-loop control system. In spite of being a difficult, yet ubiquitous and important problem, at present not enough study has been spent on the aspect of cooperative control of multi-robot nonholonomic systems subject to dynamic uncertainties and control input time-delays. The occurrence of control time-delays is primarily owe to inherent information flow delays when robots are sensing and/or communicating with spatially separated neighbors. Sufficient and necessary conditions are given in [8] for reaching an average-consensus in presence of communication time-delays. By using delayed output feedback, Qu and Wang et al. propose a sampled-data predictive cooperative control strategy for a general class of dynamic systems in [9], however, prerequisite to be input-output feedback linearizable to a given canonical form. Formation control of multi-vehicle systems in the presence of intermittent and delayed state data is demonstrated in [10] by utilizing abbreviated zero order hold scheme in conjunction with potential function method. Nevertheless, it targets only linear and holonomic systems. Although this model consisting of a single forced mass certainly simplifies the problem, it does not fully represent or precisely capture the physical system in a dynamic environment, and also leaves potential issues unaddressed that are important for practical implementation.

Cooperative Control of Multi-robot Nonholonomic Systems

479

Fig. 1 Sketch of the robot platform

v Y dr y

b

H

b

x

g

o

bo

g

l

G R0

O

X

Thereby, in this paper research emphasis is placed upon the question of how to reconsider the dynamics modeling and also control design to tackle these issues. As a direct application, the problem of coordinated trajectory tracking is discussed. The objective is to design a robust tracking controller for each perturbed nonholonomic robotic system, such that despite of the effects of uncertainties and time-delays all the closed-loop signals are bounded, the multiple independent robots can still keep a desired formation pattern, moreover the formation as a whole manages to approvingly steer themselves along sufficiently smooth time-varying spatial reference trajectories. To this end, a methodology for a class of nonholonomic nonlinear systems subject to dynamics modeling uncertainties and control input time-delays is presented. First, the system is transformed into a format favorable for control design. Second, robust tracking control realization and Lyapunov-based justification are studied. Third, a two-layer hierarchical architecture proposed in [7] is utilized to achieve multi-task cooperative control. The modeling and control strategies are finally illustrated and verified by a simulation case study. The main contribution of this paper is the presented systematic and provable design procedure for controlling collective motion of multi-robotic nonholonomic systems subject to dynamics modeling uncertainties and control input time-delays. The rest of the paper is organized as follows. Dynamics modeling and robust tracking control design for nonholonomic mobile robots are first investigated in Section 2 and 3, respectively. Simulation results are presented in Section 4. In Section 5, the paper is concluded, and possible future research directions are discussed.

2 Nominal Dynamic Model Figure 1 is a sketch of a differentially driven two-wheel robot platform. Through defining q  [Xg Yg ψ θR θL ]T , Lagrangian Principle yields the following dynamic equations of motion: M(q)q¨ + A(q, q) ˙ = C(q)T λ + Bτ , where

(1)

480

J. Zhang and S. Jayasuriya

⎡

−mw lsinψ ⎢ ψ ⎢   mw lcos  2 ⎢ M(q)  ⎢−mw lsinψ mw lcosψ 2 Id + mw d2r + I2r ⎢ ⎣ 0 0 0 0 0 0 mr 0

0 mr

⎤ 0 0 0 0⎥ ⎥ ⎥ 0 0⎥, ⎥ Ip 0 ⎦ 0 Ip

⎤ ⎡ −mw l ψ˙ 2 cosψ ⎡ ⎤ ⎢ −m l ψ˙ 2 sinψ ⎥

cosψ sinψ d2r − d2w 0 w ⎥ ⎢ ⎥ ⎢ ⎢ dr dw ⎥ and B  O3×2 , , C(q)  A(q, q) ˙ ⎢ 0 ⎥ ⎣ cosψ sinψ − 2 0 − 2 ⎦ ⎥ ⎢ I2×2 ⎦ ⎣ 0 0 −sinψ cosψ −l 0 0

where τ is the control input torques and λ is the Lagrange Multiplier. As shown in work [11, 12, 13, 7], through augmenting q into x  [q ϑ ]T  [q θ˙R θ˙L ]T , the nonholonomic system (1) can be remarkably simplified by transforming into a non-constrained format:

wherein f (x) 

x˙ = f (x) + g(x)u,

(2)

y = h(x),

(3)





E(q)ϑ O5×2 , g(x)  . O, I is zero and identity matrix, respecO2×1 I2×2

tively, and   ψ ψ ⎤ dw cos4ψ − lsin dw cos4 ψ + lsin 2d 2d r r ⎢d  sinψ + lcosψ  d  sinψ − lcosψ ⎥ ⎢ w 4 w 2dr 4 2dr ⎥ ⎢ ⎥ dw dw E(q)  ⎢ ⎥. − 2dr 2dr ⎢ ⎥ ⎣ ⎦ 1 0 0 1 ⎡

3 Robust Tracking Controller To make the problem more realistic, during the following remodeling and robust control design, modeling uncertainties and control time-delays are introduced:     x(t) ˙ = f (x(t))+ △ f (x(t)) + g(x(t))+ △ g(x(t)) u(t − τ )

(4)

y(t) = h(x(t)),

(5)

 f (x(t)) + g(x(t))u(t − τ ) + Σ (x(t), u(t − τ )),

where Σ (x(t), u(t − τ )) △ f (x(t))+ △ g(x(t))u(t − τ ). The nominal system (2)-(3) is said to have a constant relative degree ρ , if there exists 1 ≤ ρ ≤ ∞, such that Lg Lif h(x(t)) = 0 for i ∈ [0, ρ − 2] and Lg Lρf −1 h(x(t)) = 0 for x ∈ Rn , t ∈ [0, ∞)[14]. Then, there exists a neighborhood N of equilibrium point xe

Cooperative Control of Multi-robot Nonholonomic Systems

481

such that the mapping δ : N → Rn , defined as δi (x(t)) = ζi (t) = Li−1 f h(x(t)) for i ∈ [1, ρ ] and δ j (x(t)) = η j (t) for j ∈ [ρ + 1, n] satisfying Lg δ j (x(t)) = 0, is a diffeomorphism on N . The linearized nonlinear state feedback control law can be derived as −1    ρ −1 ρ − L f h(x(t)) + v(t) , u(t) = Lg L f h(x(t))

(6)

where v(t) is the newly introduced control to be designed for signal tracking. Applying the above transformation and control design to (4)-(5) yields ¯ △ Ω (x, u, u(t − τ )), ζ˙ = A¯ ζ + Bv+

(7)

η˙ = p(ζ , η )+ △ Ψ (x, u(t − τ )),

(8)

y = C¯ ζ , 

where B¯ = 0 0 · · · 1 ⎡

0 ⎢0 ⎢ ⎢. A¯ = ⎢ ⎢ .. ⎢ ⎣0 0

T

(9)

  ∈ Rρ ×1 , C¯ = 1 0 · · · 0 ∈ R1×ρ ,

⎤ 0 ⎡ ∂δ ⎤ ρ +1 0⎥ Σ (x, u(t − τ )) ⎥ ∂ x ⎢ ⎥ .. .. . . .. ⎥ .. ρ ×ρ ⎥ ⎢ ⎥ ∈ R(n−ρ )×1 and ⎦ . . . .⎥ ∈ R , △Ψ = ⎣ . ⎥ ∂ δn 0 0 · · · 1⎦ Σ (x, u(t − τ )) ∂x 0 0 ··· 0

1 0 ··· 0 1 ···

⎡

⎢ ⎢ ⎢ △Ω =⎢ ⎢ ⎢ ⎣

∂ h(x) ∂ x Σ (x, u(t − τ ))

.. .

∂ Lρf −2 h(x) Σ (x, u(t − τ )) ∂x

 ∂ Lρf −1 h(x)  Σ (x, u(t − τ )) + g(x)(u(t − τ ) − u(t)) ∂x

⎤

⎥ ⎥ ⎥ ⎥ ∈ Rρ ×1 . ⎥ ⎥ ⎦

For tracking control purpose, first define tracking error as ei = ζi − r(i−1) ∈ Rm where r is reference trajectory, and i = 1, 2, · · · , ρ . Then ¯ + B(v ¯ − r(ρ ) )+ △ Ω , e˙ = Ae

(10)

η˙ = p(ζ , η )+ △ Ψ .

(11)

Control v(t) is proposed as follows v = r(ρ ) − c1 e1 − c2 e2 − · · · − cρ eρ + ua ,

(12)

where c1 , · · · , cρ are chosen such that Sρ + cρ Sρ −1 + · · · + c2 S + c1 is a Hurwitz poly¯ in the following is controllable. nomial and also the resulted closed system (A¯ c , B) Here S is the Laplace operator. Notice that in order to compensate perturbations, an augmented robust control ua (t) is brought in.

482

J. Zhang and S. Jayasuriya

Hence, we obtain the closed-loop system, called singularly perturbed system:

where

¯ a + △ Ω (x, u, u(t − τ )), e˙ = A¯ c e + Bu

(13)

η˙ = p(ζ , η )+ △ Ψ (x, u(t − τ )),

(14)

⎡

0

1 ···

0

⎤

⎢ . .. . . .. ⎥ ⎢ . . . ⎥ . . ⎥ ∈ Rρ ×ρ . A¯ c  ⎢ ⎢ ⎥ ⎣ 0 0 ··· 1 ⎦ −c1 −c2 · · · −cρ

¯ given in system (13) is controllable, then for any symIt is known that if (A¯ c , B) metric positive definite matrix (spdm) Q ∈ Rmρ ×mρ the algebraic Riccati equation A¯ Tc P + PA¯ c + Q − γ PB¯B¯ T P = 0

(15)

has a solution P ∈ Rmρ ×mρ , which is also a spdm. Then this particular matrix is utilized to propose the following local state feedback controller: ua = −k¯ γ B¯ T Pe,

(16)

where the decentralized control gain k¯ satisfies 3 1 k¯ = + ∑ κi 2 i=1

(17)

and depends on dynamics uncertainties, which will be addressed in the following. Before stating main theorem, the following conditions are given. (C1): The trajectory planner guarantees the reference signal r(t) and its first ρ derivatives are all uniformly bounded by Br ∈ R+ , namely, ||r(i)(t)|| ≤ Br , for i = 0, 1, · · · , ρ . Then ||ζi || ≤ ||ei || + Br .

(18)

(C2): As indicated in [7], the internal dynamics η (t) in nominal system (2) is stable. Moreover, by Lyapunov converse theorem [15], assume there exists a Lyapunov function Vi (t) : Rn−ρ → R+ that for some χi > 0 (i = 1, ..., 4) satisfies

χ1 ||η (t)||2 ≤ Vi (η (t)) ≤ χ2 ||η (t)||2 ,

Vi (η (t)) p(0, η (t)) ≤ χ3 ||η (t)||2 , η (t) Vi (η (t)) || || ≤ χ4 ||η (t)||2 . η (t)

(19) (20) (21)

(C3): For uncertainty terms △ Ω and △ Ψ , unlike most work on control of timedelay systems has matched conditions as prerequisite, here assume less conservative mismatched conditions. Suppose there exist κi , εi ∈ R+ (i = 1, 2, 3) such that

Cooperative Control of Multi-robot Nonholonomic Systems

483

Fig. 2 A framework for cooperative control

Jd -

6

+

/

Trajectory Planner

J

P d = fˆ ( J , J d )

( P d , P d ) -

K

6

+

x = f ( x) + g ( x)u

Tracking Controller

y

||2P △ Ω (x, u, u(t − τ ))|| ≤ κ1 ||ζ || + κ2 ||η || + κ3 , || △ Ψ (x, u(t − τ ))|| ≤ ε1 ||ζ || + ε2 ||η || + ε3 ,

(22) (23)

where P is defined in (15). Moreover, if p(ζ (t), η (t)) is Lipschitz in ζ (t), then it is known that the following holds: ||p(ζ (t), η (t)) − p(0, η (t))|| ≤ L||ζ (t)||, ∀ η (t) ∈ Rn−ρ ,

(24)

where L ∈ R+ , a Lipschitz constant of p(ζ (t), η (t)) with respective to ζ (t). Theorem 1. For the uncertain nonlinear system (4)-(5) with control input timedelays satisfying conditions (C1)-(C3), under control law (12) and (15)-(17) there exist ς ∗ and ε ∗ for all 0 < 21 + κ1 ≤ min( λmin2(Q) , ς ), κ2 ∈ (0, ς ∗ ] and ε2 ∈ (0, ε2∗ ] such that the states of the closed system (13)-(14) are uniformly bounded. Furthermore, the output tracking error e(t) converges to a residual set

Γe = {e(t) ∈ Rmρ |V (e(t)) ≤ ke−1 be } where ke 

λmin (Q) − κ1 − 12 and be  (κ2 Bi )2 + (κ1Br + κ3 )2 . λmax (P)

Proof. To streamline the readability, the proof is placed in the Appendix.

⊓ ⊔

As a demonstration and verification of the presented modeling and control schemes for the nonholonomic robotic system, the problem of coordinated trajectory tracking is considered. We employ our previously developed two-stage hierarchical architecture for cooperative control (Fig. 2) and incorporate the above dynamics modeling and robust tracking control design into the bottom tracking controller layer. The top trajectory planner generates motion reference trajectories by utilizing a null-space method [16, 17] to integrate and prioritize the desired multiple tasks in a systematic fashion, which dynamically uses real-time local sensing and/or communication

484

J. Zhang and S. Jayasuriya

information. For more details and discussions about the proposed framework, please refer to [7].

4 Case Study The lumped nonlinear perturbation in system (4) is modeled as Σ (x(t), u(t − τ ))   T ϖ1 ϖ1 ϖ2 [|1 − sinψ | θ˙R + θ˙L cosψ θ˙R + sinψ θ˙L ] u(t − τ )W , where ϖ1 and ϖ2 are un-

certain

that randomly lie within [-0.03 0.03] and [-0.1 0.1], respectively; parameters ϖ2 0 . The output h(x)  [Xg ,Yg ]T is used for trajectory tracking. Sampling 0 |cosψ | period is △ t = 0.1 sec and τ = 6 △ t .

W

100

80

60

40

Y (m)

20

0

−20

−40

−60

−80

−100

−50

0 X (m)

50

100

50

100

(a) 100

80

60

40

Y (m)

20

0

−20

−40

−60

Fig. 3 Robots track a circle avoiding inter-agent collision and external obstacle (a) without robust control ua and (b) with ua

−80

−100

−50

0 X (m)

(b)

Cooperative Control of Multi-robot Nonholonomic Systems

485

18 d12 d13 d23 16

d (m)

14

ij

12

10

8

6 0

0.5

1

1.5

2

2.5 t (sec)

3

3.5

4

4.5

5

Fig. 4 Internal distances between robots 15

2

uR (deg/sec )

10 5 0 −5 −10

0

0.2

0.4

0.6

0.8

1 t (sec)

1.2

1.4

1.6

1.8

2

0

0.2

0.4

0.6

0.8

1 t (sec)

1.2

1.4

1.6

1.8

2

10

2

uL (deg/sec )

5

0

−5

−10

Fig. 5 Delayed control inputs

In this case study, from any initial distribution, multiple robots are required to arrange themselves into a desired regular polygon pattern and track a circular trajectory while avoiding inter-agent collisions and obstacles in the environment. As indicated in Fig. 3(a), without augmented robust control ua , the desired formation mission cannot be well achieved. However, by including ua and with selected robust control gains1 : γ  1000, c1  2025, c2  63 and Q  500I4×4 , figure 3(b) shows three robots that are randomly posed around origin area (plus sign) successfully accomplish the tracking circle task (dashdot line) without any inter-agent collision, even during the transient phase before stabilizing into formation. Furthermore, an external obstacle (solid line) is avoided when it is present. Figure 4 gives the internal distances between robots, which illustrates that the three robots, while tracking the 1

The other parameters of the robotic system and trajectory planner are adopted the same with [7].

486

J. Zhang and S. Jayasuriya

circular reference trajectory, approximately form an equilateral triangle in spite of the existing of perturbations. Note that even when avoiding the obstacle the internal distances are bounded within a certain range: The lower bound prevents collision and the upper bound constrains the formation to be in cohesion and guarantees no robot would escape from the team. Smooth curves in Fig. 5 are the transients of the right and left delayed control inputs of each robot. This satisfactory performance demonstrates the effectiveness of the above robust tracking control design.

5 Conclusion and Future Work To address the uncertainty and time-delay issues in cooperative control of multirobot nonholonomic systems, dynamics modeling and Lyapunov-based local robust tracking control are investigated to effectively suppress these perturbations and ensure control performances. By integrating them into the two-layer framework proposed in [7], the effectiveness of the modeling and control strategies are further validated through the successful fulfillment of the desired multi-robot behaviors. However, It needs pointing out that for this high gain design to get satisfactory performances the time-delays are required to be sufficiently small. For large timedelays, time-advanced state prediction is necessary to maintain performances. A nonlinear state predictor is being investigated and adopted to estimate the future states so that traditional controllers can still be applied to accomplish given tasks in spite of the appearance of large time-delays.

References 1. Strogatz, S.H.: Exploring complex networks. Nature 410, 268–276 (2001) 2. Vicsek, T., Czirok, A., et al.: Novel type of phase transition in a system of self-driven particles. Phys. Rev. Lett. 75(6), 1226–1229 (1995) 3. Guessoum, Z., Rejeb, L., Durand, R.: Using adaptive multi-agent systems to simulate economic models. In: Kudenko, D., Kazakov, D., Alonso, E. (eds.) AAMAS 2004. LNCS, vol. 3394. Springer, Heidelberg (2005) 4. Cao, Y.U., Fukunaga, A.S., Kahng, A.: Cooperative mobile robotics: antecedents and directions. Autonomous Robots 4(1), 7–27 (1997) 5. Parker, L.E.: Current state of the art in distributed autonomous mobile robotics. In: Distributed Autonomous Robotic Syst., vol. 4, pp. 3–12. Springer, Heidelberg (2000) 6. Murray, R.M.: Recent research in cooperative control of multi-vehicle systems. Trans. ASME, J. Dyn. Syst. Meas. Control 129(5), 571–583 (2007) 7. Zhang, J., Yi, J., Jayasuriya, S.: A framework for multi-task formation control of nonholonomic robotic systems. In: ASME Dyn. Syst. and Contr. Conf., October 20-22, Ann Arbor, MI, USA (2008) 8. Olfati-Saber, R., Murray, R.M.: Consensus problems in networks of agents with switching topology and time-delays. IEEE Trans. on Autom. Contr. 49(9), 1520–1533 (2004) 9. Qu, Z., Wang, J., Hull, R.A., Martin, J.: Cooperative control design and stability analysis for multi-agent systems with communication delays. In: IEEE Int. Conf. on Robot. and Autom., pp. 970–975 (2006)

Cooperative Control of Multi-robot Nonholonomic Systems

487

10. Triplett, B., Morgansen, K.: Abbreviated zero order hold for formation control in the presence of communication and sensing delays. In: American Contr. Conf., pp. 6081– 6087 (2006) 11. Pars, L.A.: A Treatise on Analytical Dynamics. Ox Bow Press (1979) 12. Amirouche, F.M.L.: Computational Methods in Multibody Dynamics. Prentice-Hall, Englewood Cliffs (1992) 13. Bloch, A.M., Reyhanoglu, M., McClamroch, N.H.: Control and stabilization of nonholonomic dynamic systems. IEEE Trans. on Autom. Contr. 37(11), 1746–1757 (1992) 14. Isidori, A.: Nonlinear Control Systems, 3rd edn. Springer, Heidelberg (1995) 15. Khalil, H.K.: Nonlinear Systems, 3rd edn. Prentice-Hall, Englewood Cliffs (2001) 16. Antonelli, G., Chiaverini, S.: Kinematic control of platoons of autonomous vehicles. IEEE Trans. on Robotics 22(6), 1285–1292 (2006) 17. Antonelli, G., Arrichiello, F., Chiaverini, S.: Experiments of formation control with collisions avoidance using the Null-Space-based Behavioral control. In: 14th Mediterranean Conf. on Contr. and Autom. (2006)

Appendix: Proof of Theorem 1 Proof. To investigate the boundedness, define the following Lyapunov function V (e(t), η (t)) = Ve (e(t)) + ς Vi (η (t))  eT (t)Pe(t) + ς Vi(η (t)),

(25)

where P satisfies (15) and ς ∈ R+ . Notice function Vi (η (t)) is added to ensure the stability of internal dynamics. Differentiating V (t) along the trajectory of closed system (13)-(14) yields V˙ =e˙T Pe + eP˙eT + ς V˙i ¯ a+ △ Ω ) ¯ a + △ Ω )T Pe + eT P(Ac e + Bu =(Ac e + Bu   Vi p(ζ , η )+ △ Ψ . +ς η

(26)

With the aid of (15),   ¯ a + 2eT P △ Ω + ς Vi p(ζ , η )+ △ Ψ V˙ =eT (−Q + γ PB¯B¯ T P)e + 2eT PBu η T ¯ 2 T T ¯ = − e Qe + γ (1 − 2k)||e PB|| + 2e P △ Ω  Vi  p(0, η ) + p(ζ , η ) − p(0, η )+ △ Ψ . (27) +ς η Under (17), it follows that  Vi  p(0, η ) + p(ζ , η ) − p(0, η )+ △ Ψ . V˙ ≤ − eT Qe + 2eT P △ Ω + ς η Based on Rayleigh Principle2 , (C2) and (C3), 2

λmin (Q)||e||2  eT Qe  λmax (Q)||e||2

(28)

488

J. Zhang and S. Jayasuriya

V˙ ≤ − λmin (Q)||e||2 + ||e||(κ1 ||ζ || + κ2||η || + κ3 )

− ς χ3 ||η ||2 + ς χ4 L||η ||||ζ (t)|| + ς χ4 ||η ||(ε1 ||ζ || + ε2 ||η || + ε3 ).

(29)

Under the constraint of external dynamics (18), (29) can be further formulated into   V˙ ≤(κ1 − λmin (Q))||e||2 + (κ1Br + κ3 )||e|| + κ2 + ς χ4 (ε1 + L) ||e||||η ||   + ς (ε2 χ4 − χ3 )||η ||2 + ς χ4 ε3 + Br (ε1 + L) ||η ||. (30) Since

1 (κ1 Br + κ3)||e|| ≤ ||e||2 + (κ1 Br + κ3)2 4    2 1 κ2 + ς χ4 (ε1 + L) ||e||||η || ≤ ||e||2 + κ2 + ς χ4(ε1 + L) ||η ||2 4  χ ||η ||2 χ 2 ε + B (ε + L)2    r 1 3 3 + 4 , ς χ4 ε3 + Br (ε1 + L) ||η || ≤ ς 4 χ3  3  2  1 ||η ||2 V˙ ≤ − (λmin (Q) − κ1 − )||e||2 − ς ( χ3 − χ4 ε2 ) − κ2 + ς χ4 (ε1 + L) 2 4  2 ς χ42 ε3 + Br (ε1 + L) 2 . (31) + (κ1Br + κ3 ) + χ3 Let ς ∗ 

χ3



12 1+ χ4 (ε1 +L)

2 and ε2∗ 

χ3 χ4 ,

then the hypotheses of the Theorem 1 imply

λmin (Q) ς ∗ χ3 V˙ ≤ − ||e||2 − ||η ||2 + d0 , 2 2  2 ς χ42 ε3 + Br (ε1 + L) 2 . where d0  (κ1 Br + κ3 ) + χ3

(32)

∗

If k0  min( λmin2(Q) , ς 2χ3 ), then V˙ ≤ −k0 (||e||2 + ||η ||2 ) + d0, which says for sufficiently large ||e(t)|| or ||η (t)||, the states of closed system (13)-(14) are uniformly bounded for all t ≤ 0. To investigate the ultimate bound of tracking errors, inequality (31) yields: 1 V˙e ≤ −(λmin (Q) − κ1 − )||e||2 + κ2||η ||2 + (κ1Br + κ3 )2 . 2

(33)

Suppose internal dynamics are constrained as ∃ Bi ∈ R+ , such that ||η || ≤ Bi , then V˙e ≤ −

(λmin (Q) − κ1 − 21 ) V + (κ2Bi )2 + (κ1 Br + κ3 )2 ≤ −keV + be , λmax (P)

which implies Ve (e(t)) ≤ Ve (e(t0 ))e(−ket) + (1 − e(−ket) )ke−1 be .

(34)

Cooperative Control of Multi-robot Nonholonomic Systems

489

Hence, the output tracking error e(t) will eventually converge to the residual set Γe ,

Γe  {e(t) ∈ Rmρ |V (e(t)) ≤ ke−1 be }. Clearly, the size of the compact set relies on the system uncertainties and design parameters. This completes the proof. ⊓ ⊔

Guaranteed-Performance Multi-robot Routing under Limited Communication Range Alejandro R. Mosteo, Luis Montano, and Michail G. Lagoudakis

Abstract. A common assumption made in multi-robot research is that robots operate in a fully networked environment, where any two robots can communicate reliably at any time. However, in real-world missions over large geographical areas robots have a limited communication range and signal propagation conditions may be unpredictable due to obstacles or interferences. In such cases, communication limitations must be taken explicitly into account. In this paper, we present a multirobot routing algorithm for dense environments under limited communication range. We focus on the aspect of careful mission planning in order to guarantee continuous connectivity and mission completeness, while offering a guarantee on performance under a variety of team objectives. The proposed algorithms are demonstrated in a large-scale domain using realistic simulations on the Player/Stage platform.

1 Introduction Many application domains involve tasks of visiting places of interest using robots: soil or atmospheric sampling, sensory inspection, object delivery, etc. This problem of routing a team of robots over appropriate paths with the goal of visiting all target locations (corresponding to specific tasks) is known as multi-robot routing. A solution to this problem guarantees that all locations are visited, while some team performance measure of cost is optimized. It is known that solving multi-robot routing optimally is an NP-hard problem for many interesting team objectives [4]: Alejandro R. Mosteo and Luis Montano Instituto de Investigaci´on en Ingenier´ıa de Arag´on, University of Zaragoza, 50018 Zaragoza, Spain e-mail: {amosteo,montano}@unizar.es Michail G. Lagoudakis Department of Electronic and Computer Engineering, Technical University of Crete, 73100 Chania, Crete, Greece e-mail: [email protected]

492

A.R. Mosteo et al.

minimizing total mission cost (energy), minimizing total mission time (makespan), and minimizing average service time (latency). There is a growing trend to drop the assumption that the robots of the team are able to freely communicate since, in many interesting real-world applications, network connectivity must be explicitly considered and preserved. Furthermore, while some domains may tolerate temporary losses of communication between team members, in others it is mandatory to maintain connectivity at all times: monitored surveillance, rescue missions, tightly coordinated cooperation, etc. These include domains where uninterrupted contact to a stationary monitoring base or command center is commonly required. Mobile robots typically use an ad-hoc wireless connection to communicate. Maintaining full connectivity between team members does not necessarily imply that each robot communicates directly with all other robots, but rather that any robot can reach any other robot either directly or by relaying messages through some other robot(s). Therefore, it is required at all times that a spanning tree of network links over all robot locations exists. Adding such limited communication constraints to multi-robot routing makes the problem even harder. In this paper we present an approach tailored to these concerns which provides a routing solution suitable for dense environments and prevents network splits with minimal assumptions on the signal propagation. In effect, each target is eventually served by a group of robots without breaking their connectivity requirement to a stationary base. We study algorithms that yield efficient routes and we prove that the quality of these routes is within a factor from the optimal ones using graphtheoretic arguments. The proposed approach is validated in a large realistic scenario using the Player/State robot simulator. The paper is organized as follows: Section 2 reviews the issue of limited connectivity and how other researchers have addressed it. Section 3 provides a formal definition of the multi-robot routing problem. Section 4 describes the proposed strategy, whose theoretical properties and quality bounds are proved in Section 5. Empirical results from simulation are shown and discussed in Section 6. Finally, we discuss future work and conclude in Section 7.

2 Limited Connectivity The limited connectivity issue is a growing concern in multi-robot research. One common approach is to include link quality in the motion [7] or task generation functions. This idea is not readily applicable to general-purpose field or service robotics, because the tasks are provided by external sources and may be totally unrelated to those generated by the team in order to preserve communication. These approaches are, furthermore, prone to local minima problems where some robots become trapped by obstacles, threatening mission completion. To work around this problem, some proposals [10] require temporary network splits to allow for escape routes, and therefore cannot guarantee uninterrupted connectivity.

Guaranteed-Performance Multi-robot Routing

493

General approaches suitable for general-purpose service robotics, where tasks are not tied to a particular problem domain and uninterrupted connectivity is an explicit requirement, are scarce. In [3], a line-of-sight constraint is applied, but actual link quality is not used. Other approaches rely heavily on assumptions about the signal decay function. However, there is growing evidence and concern [9] suggesting that such models can badly misrepresent the behavior of real signals, leading to temporary connectivity losses or outright algorithm failures. Real signals in populated scenarios exhibit a complex behavior caused by reflections, multi-paths, scattering and interferences, among other factors [11]. This uncertainty directed us towards reactive solutions where the actual signal is measured in situ, instead of relying on a predictive model. Along these lines, in the past we proposed several algorithms [6] that operate on top of a signal-constrained navigation model, albeit only in an obstacle-free environment. An alternative proposal experiments with a robot that deploys stationary relays when necessary to extend its range of operation [8] in out-of-sight scenarios, focusing mainly on hardware issues. In this work we extend ideas found in both directions of this recent work to provide a general solution for field robots in scenarios with arbitrarily shaped obstacles. Reactive operation based on measured signal enables us to drop most assumptions on the signal propagation model. In practice, we only require that the decay funcFig. 1 Typical degradation of the radio signal tion is reasonably continuous, meanquality as a function of the distance between ing that, between two consecutive mea- transmitter and receiver. A link is broken when surements of signal quality made by a signal quality drops to the forbidden zone moving robot, the signal will not skip a controlled zone that serves as a buffer between the safe and the forbidden zones (Fig. 1). As a bonus, our approach in this work is readily applicable to tethered robots [5] and teams of robots.

3 Multi-robot Routing The considered multi-robot routing problem is formally specified by a set of robots, R = {r1 , r2 , . . . , rn }, a set of targets, T = {t1 ,t2 , . . . ,tm }, the locations of both robots and targets on the two-dimensional plane, and a non-negative cost function c(i, j), i, j ∈ R ∪ T , which denotes some abstract cost of moving between locations i and j (e.g., distance, energy, time, etc.). We assume that the robots are identical; therefore the same cost function applies to all robots. Typical cost measures are travel distance, travel time, or energy consumption between locations. In this work, we additionally assume that the cost function satisfies the triangle inequality. This assumption holds as long as the robots follow the cheapest obstacle-free path when moving between locations.

494

A.R. Mosteo et al.

The mission begins with all robots located at some base location and completes when all targets are visited and the robots return to base. The problem of multirobot routing is to find an allocation of targets to robots and a path for each robot that begins at the base, visits all targets allocated to it, and ends back to the base, so that the mission is completed and a team objective is minimized. Three intuitive team objectives [4], corresponding to minimizing energy, makespan, and latency respectively, are: M IN S UM: Minimize the sum of the robot path costs over all robots. M IN M AX: Minimize the maximum robot path cost over all robots. M INAVE : Minimize the average target path cost over all targets.

The robot path cost of a robot is the sum of the costs along its entire path. The target path cost of a target t is the total cost of the path traversed by the designated robot from the base up to target t along its path. The three team objectives above can be formally expressed as min ∑ RPC(r j , A j ), A

j

min max RPC(r j , A j ), A

j

min A

1 CT PC(r j , A j ), m∑ j

where A = {A1 , A2 , . . . , An } is a partition of the set of targets, so that targets in Ai are allocated to robot ri , RPC(ri , Ai ) denotes the robot path cost for robot ri to visit all targets in Ai starting from and finishing to the base, and CT PC(ri , Ai ) denotes the cumulative target path cost of all targets in Ai , again, if robot ri visits all targets in Ai starting from the base.

4 The C ONNECT T REE Algorithm Our generic strategy for multi-robot routing under limited communication range in dense environments is based on the following ideas. Robots move together as a whole towards a target, monitoring the network link to the base; once this link falls under the safety threshold and into the controlled zone, one robot is stopped at that point to act as a relay, and the rest continue advancing while monitoring now their link to the stopped relay robot. Using this pattern repeatedly, the robots can reach far away from the base without ever breaking the link to the base. Note that the robots do not need to know their communication range; the decision to stop a robot as a relay is made dynamically as dictated by the locally measured signal quality. Once the target is visited, the robots are allowed to retreat from the target only along the path they used to reach it to avoid potential deadlocks due to the density of obstacles and the connectivity constraint. This retreat mirrors the deployment, and a relay can only start retreating when all moving robots reach the relay location. In practice, the set of paths used by the robots is a tree rooted at the base station (Fig. 2). Note that two paths to different targets may share a portion in the upper levels of the tree. Therefore, after visiting a target, robots do not need to retreat all the way to the base to move on to the next target. Instead, it is sufficient to retreat up to

Guaranteed-Performance Multi-robot Routing

495

a common ancestor point between the two paths. Thus, during the entire process, critical network links lie within the tree and the relative robot ordering along any path remains the same when going towards a target and when retreating back. This would allow the use of this approach in conveniently tethered robots, as well as in a single robot capable of deploying and collecting static relays. Alg. 1 shows the pseudo-code for the C ONNECT T REE algorithm. It can be argued that it is more M IN S UM-cost efficient to begin moving only one robot, and move relays one by one on demand once a weak signal is detected between any robot pair. This scheme is not recommended for completeness purposes; it involves two moving endpoints in the network links, instead of just one, increasing the risk of sharp network dropouts. Moreover, it Fig. 2 A possible path tree visitdoes not improve M IN M AX and M INAVE. There are two key subroutines affecting per- ing four goals. Any depth-first traversal gives minimum backtracking and formance in the C ONNECT T REE algorithm. Althus mission timespan. However, 3locateFrom (line 2) decides the next target to 2-1-4 would minimize average cuvisit. One advantage of using a tree structure is mulative cost. the fact that any depth-first traversal will have minimum backtracking along the branches, and thus gives the least M IN S UM and M IN M AX cost for the given tree. We used depth-first traversals in this work, while future research may specifically address traversal orderings for the M INAVE objective. The most influential subroutine, however, is BuildTree (line 1), which builds the tree routes of the robots. Depending on the depth of the tree from the base, the robots may or may not be able to traverse it in its entirety. If the depth exceeds the maximum range they can reach collectively, the remaining target(s) along that branch will be abandoned. To avoid such potential failures, one may use a straightforward scheme: construct a star-like tree where each target is attached directly to the root (base) using the least-cost path between them. Despite its simplicity and the potential inefficiency of the resulting tree, this scheme guarantees that the mission will be accomplished, if it can be accomplished. No target will be abandoned, unless it is beyond the ability of the team to reach it. We refer to this simple variation of the C ONNECT T REE algorithm as STAR. Further improvements studied in this paper focus solely on tree construction with the goal of achieving provably good performance. It should be clear that the total cost of the constructed tree is indicative of the team performance during the mission. Additionally, the depth of the tree plays a key role in the success of the mission. Therefore, it makes perfect sense to try to keep the total cost and the depth of the tree low. Towards this end, we suggest three different ways of building such trees based on solving a Traveling Salesman Problem (TSP), finding a Minimum Spanning Tree (MST), and finding a depth-limited spanning tree, respectively. TSPT: The first proposed algorithm is based on computing or approximating an optimal open TSP tour, which visits all targets beginning from the base exactly once,

496

A.R. Mosteo et al.

Algorithm 1. Description of the C ONNECT T REE strategy. Variable i defines the range i..n of moving robots, while 1..i − 1 represents the stopped relays. r0 , for simplicity, denotes also the Base. ωi, j denotes link quality between robots i, j (or Base and r j for ω0, j ). UT is the upper threshold of the controlled zone. Variations are denoted by s ∈ S = {STAR, TSPT, MST, DLST} Inputs: Base, T , n, s 1: PathTree ← BuildTree(s, Base, T ) // Construct the initial tree of paths. 2: CurrentTarget ← AllocateFrom(PathTree) // Choose first target. 3: i ← 1 // Initially, all robots move. 4: while PathTree = 0/ do // Main mission loop, targets remaining. 5: while ¬Reached(CurrentTarget) do // Advancing towards target. 6: StepTowards(who ⇒ {ri . . . rn }, where ⇒ CurrentTarget) 7: if ωi−1,i ≤ UT then // Link exhausted, ri must stop. 8: if i = n then // Was ri our last available robot?. 9: PathTree ← BuildTree(s, Base,T ∩RemainingTasksIn(PathTree)) 10: exit // Tree rebuilt, backtrack to new path. 11: else // Stop ri to act as relay. 12: StopRobot(i); i ← i + 1 13: end if 14: end if 15: end while 16: if Reached(CurrentTarget) then 17: PathTree ← PathTree \ CurrentTarget // Remove reached target. 18: end if 19: if PathTree = 0/ then // Select next target from remaining ones. 20: NextTarget ← AllocateFrom(PathTree) 21: BackPoint ← CommonAncestor(CurrentTarget,NextTarget) 22: // Deepest common node in the retreat path and the new path. 23: else 24: NextTarget ← BackPoint ← Base // Only remains the comeback to base. 25: end if 26: while ¬Reached(BackPoint) do // Retreating from current target. 27: StepTowards(who ⇒ {ri . . . rn }, where ⇒ BackPoint) 28: if Reached(ri−1) then // Retreat starts for deepest stopped relay. i ← i−1 29: 30: end if 31: end while 32: CurrentTarget ← NextTarget // Start advancing to next target. 33: end while

but does not return to the base. Despite the NP-hardness of the problem, there are several efficient algorithms and heuristics for finding optimal or near-optimal TSP tours for real-world problems in reasonable time. An open TSP tour is a degenerate tree with just a single branch, therefore C ONNECT T REE can easily traverse it.

Guaranteed-Performance Multi-robot Routing

497

However, its depth may be well beyond the reaching ability of the team and the mission may fail even though the robots might have the ability to accomplish it. To guarantee mission completeness, we propose an iterative replanning scheme: As soon as an unreachable target is detected during tree traversal, a new open TSP tour is constructed from the base to the remaining non-visited targets. The robots will retract back, in the worst case all the way to the base, to traverse the new tree. The first target on the new tree will be connected to the base with the least-cost path, therefore at least one target will be visited at each replanning and the mission will progress steadily. Under this replanning scheme, the mission will always be completed, if it can be completed. We refer to this variation of the C ONNECT T REE algorithm as TSPT (TSP tree). MST: The second proposal is based on finding a minimum spanning tree (MST) over all target nodes and the base. Using Prim’s algorithm with the base as the seed node, a minimum spanning tree is easily constructed in polynomial time and can be traversed by the C ONNECT T REE algorithm. Even though its total cost is kept low, the depth can be arbitrary, but certainly no more than the depth of the TSP tree. Again, to guarantee mission completeness, we adopt the same replanning scheme. As before, the mission will always be completed, if it can be completed. We refer to this variation of the C ONNECT T REE algorithm as MST. DLST: Our last proposal is based on finding a depth-limited spanning tree with the goal of minimizing the possibility of a costly replanning step. If the robots have some estimate of the maximum depth D they can collectively reach, then we can construct a spanning tree whose depth will not exceed D. This is achieved by a simple modification to Prim’s algorithm: at each step we only consider those edges between the partial tree and the remaining nodes that do not result in a tree depth greater than D. In a situation of abandoned targets due to an optimistic estimate of D, we proceed with the replanning scheme as above. We refer to this variation of the C ONNECT T REE algorithm as DLST (Depth-Limited Spanning Tree). In summary, all four variations of the C ONNECT T REE algorithm are safe (network integrity is maintained at all times) and complete (mission is successfully accomplished) for any number of robots, any number of targets, and any communication range per robot, unless the mission is a failure by design (some target is out of range even using the optimal path with all robots).

5 Performance Bounds From a graph-theoretical point of view, the problem of finding the “best” tree to be traversed by C ONNECT T REE is equivalent to a minimum Steiner tree problem [2]: given a weighted graph G = (V, E, w) and a subset U ⊆ V , find a tree in G that interconnects all vertices in U with the least cost. Notice the difference to an MST problem over U; a Steiner tree may contain additional vertices not in U that help reduce the cost. The Euclidean Steiner tree problem considers the interconnection of |U| points on the plane with the minimum cost using any intermediate points on

498

A.R. Mosteo et al.

the plane. In our case, the Steiner tree would be rooted at the base with all targets at the leaves. The minimum Steiner tree problem is known to be NP-hard, therefore we abandoned the idea of finding an optimal Steiner tree in favor of approximation algorithms. In this section, we analyze the proposed algorithms and relate their performance to a baseline performance determined as the M IN M AX cost (or the M IN S UM cost; they are the same) of a closed TSP tour of a single robot that begins at the base, visits all targets, and returns to the base without any communication range limits. We refer to this ideal solution as CTSP (Closed TSP) and we define the performance ratio of any algorithm as the ratio of its performance according to some metric over the baseline performance. Our main results are stated below. Theorem 1. The performance ratio of the TSPT and MST variations of C ONNECTT REE without replanning is at most 2n for the M IN S UM, at most 2 for the M IN M AX, and at most 2 for the M INAVE objectives.1 Proof. Under our tree traversal conventions, the mission cost according to the M IN M AX metric of any variation will be exactly twice the total cost of the underlying tree. Clearly, the open TSP tour cannot be worse than the closed TSP tour and the M IN M AX performance of TSPT is exactly twice the cost of an open TSP tour. Therefore, M IN M AX(TSPT) ≤ 2 M IN M AX(CTSP) and the M IN M AX performance ratio of TSPT is bounded by 2. Similarly, it is known that in metric spaces, like ours, the total cost of a minimum spanning tree does not exceed the cost of a closed TSP tour. The M IN M AX performance of the MST variation is exactly twice the cost of a minimum spanning tree. Therefore, M IN M AX(MST) ≤ 2 M IN M AX(CTSP), and the M IN M AX performance ratio of MST is also bounded by 2. The M IN S UM cost of any variation cannot be determined precisely, however it will be within a factor n of the M IN M AX metric (all robots traverse the entire tree in the worst case), that is M IN S UM(X) ≤ n M IN M AX(X). Similarly, the M INAVE metric for any variation will not exceed the M IN M AX metric (almost all targets are placed at the leaf of the tree visited last in the worst case, therefore each target is visited just before the end of the mission), that is M INAVE(X) ≤ M IN M AX(X). The remaining bounds are now obvious. Theorem 2. The performance ratio of the TSPT and MST variations of C ONNECTT REE with replanning is at most 2nm for the M IN S UM, at most 2m for the M IN M AX, and at most 2m for the M INAVE objectives. Proof. Each planning or replanning of a tree eliminates at least one target, therefore in the worst case m trees will be build. For each of these m trees the previous bounds hold, therefore it suffices to include a factor of m. Despite the fact that these results represent a worst-case, pessimistic analysis that yields loose bounds, it is striking that the additional cost introduced by the limited 1

As of this time, we have not been able to obtain bounds on the performance of DLST, but we are confident they exist.

Guaranteed-Performance Multi-robot Routing

(a) STAR

499

(b) MST

Fig. 3 Simulation scenario with overlaid path trees. Twenty targets are shown for clarity, although fifty targets were actually used in simulations

communication range cannot really grow arbitrarily. If the need for replanning is avoided, then performance is guaranteed to be very close to the ideal CTSP performance. Our experimental results demonstrate that performance in practice is well below the theoretical bounds.

6 Simulation Results The simulation map is a 135m × 86m realistic scenario (Fig. 3), modeled after a shopping mall close to our lab, where obstacles are the product stands. This scenario is interesting, not only because of the amount, size and density of obstacles, but also because of the large central obstacle: Its particular configuration and its relative position to the deployment point would trap robots in deadlock situations (which cannot happen in our approach) when using methods based on local utility functions. A collaboration agreement has been reached with the owners of the shopping mall to conduct real experimentation on-site in the near future. Visiting randomly distributed targets in such a scenario may represent tasks of object inspection or delivery, service requests, or sample collection missions, for example. The simulation environment was the Player/Stage robot simulator [1] which offers realistic mobile robot kinematics. The robot team consisted of nine simulated Pioneer 3AT robots, one of which served as the communication base and stayed at the initial point of deployment. High-level path planning was achieved by applying standard A∗ search to an 8-vicinity grid of fixed size over the mission area. Paths obtained this way were used for constructing trees over the grid. There were n = 8 mobile robots and m = 50 targets in the mission. The targets were uniformly randomly distributed over the free cells of the map. The communication range was modeled as a circular area of fixed radius around the robot. The value of this radius was not known to the robots at execution time. Instead, they queried a function that returned a simulated measurement of link quality (ωi, j in Alg. 1) derived from this radius.

500

A.R. Mosteo et al.

L=25m

16000

L=50m 14000

L=Inf.

MinSum (m)

12000 10000 8000 (x8) 6000 4000 2000 0

ST

AR

TS

PT

T

MS

DL

ST

ST

AR

PT ST T S T/DL MS

CT

SP

Fig. 4 Left: M IN S UM evaluation. Right: M IN M AX evaluation

More specifically, the radius length L was set to the values of L = 25m and L = 50m, typical of wifi transmitters. The shortest range is barely enough to reach the farthest places in the map, thus making the abandonment of targets a significant concern, and was chosen to gain information on performance under extreme conditions. The longest range, on the contrary, provided insight on the improvements that can be gained, when Fig. 5 M INAVE evaluation we are not too tight on resources. For the DLST variant, we set D = nL. Finally, the CTSP cost of the ideal closed TSP tour was measured to serve as a comparison basis with respect to the theoretical bounds; CTSP was obtained using a single robot without any network restrictions. Ten runs were performed for each algorithm and each value of L, using a different seed for each run; nevertheless, the same set of seeds was used across the different algorithms, so that they are compared on the same basis. Fig. 3 shows two initial trees for 20 targets only (for clarity). It can be easily seen that the STAR tree is more balanced than the MST tree; the MST tree eventually might require a replanning to complete the mission. Figs. 4 to 5 show mean values and the 95% confidence intervals for each of the three performance metrics in all cases. We observed that DLST with L = 50 generated the same solutions as MST. This means that the branch depth limit was not hit, and thus both are conflated in the figures. We highlight the following observations: • Traversal of the STAR tree requires a noticeable additional amount of backtracking, which results in the worst performance in all cases. • MST is the best performer for M IN M AX and M INAVE. Of special interest is the fact that it outperforms DLST for L = 25, despite DLST making use of the D limit. This is a consequence of the pessimistic prediction of DLST: paths

Guaranteed-Performance Multi-robot Routing

•

•

•

•

•

501

along the branches of the tree are usually longer than the Euclidean distance between stopped relay robots (the signal propagates in straight lines). Consequently, DLST discarded branches during tree construction that were in fact valid. In the M IN S UM case, we observed that TSPT and MST performed comparably well. Together with the previous point, this makes MST the best choice among the tested strategies. Comparing the performance of the algorithms to the ideal CTSP baseline performance (shown only in Fig. 4–Right), we see that all instances with known bounds are well below these bounds. Thus, the typical performance is noticeably better; the worst case represents rather pathological cases. Note that the M IN S UM performance is specially affected in the C ONNECT T REE strategy; we could say that it is its “Achilles’ heel” and the price it pays for network integrity. Since all robots not acting as static relays are moving together all the time, this causes the M IN S UM cost to grow by a factor of n in the worst case. To allow for a fair comparison, we have plotted the actual M INAVE cost of CTSP in Fig. 5 instead of the baseline cost. Similarly, in Fig.4, the baseline CTSP cost would correspond to a single robot performing the closed route. However, to make the comparison more meaningful, in this figure the CTSP cost is n times that of the single robot. We did not run the algorithms with L = ∞ because no replannings were necessary for L = 50. Consequently, we would have observed the same M IN M AX and M INAVE costs, and an even higher M IN S UM cost.

7 Conclusion In this paper we studied the problem of multi-robot routing under communication constraints, when network integrity is a primary concern and dense obstacles make the modeling of accurate signal propagation too difficult. The proposed strategy relies on the careful construction of a tree of paths, rooted at a monitoring station, that reaches all targets. Robots traverse this tree in a network-safe manner which requires only local link information. A valid target may require any number of robots (even all) to be serviced, even using the optimal route; all such targets are guaranteed to be serviced. We focused on tree construction algorithms backed with theoretical bounds on their performance with respect to a baseline performance of an optimal closed TSP tour cost in a network unrestricted environment. Four algorithms were studied: a simple star tree of least-cost paths from the base to each target, an optimal open TSP tour, a minimum spanning tree, and a depth-limited spanning tree. Simulations in a large realistic scenario empirically demonstrated improvement over the simple star solution, and identified the minimum spanning tree (MST) as the best choice for our network safe approach. We also observed that the empirical performance ratios are well below the worst case bounds.

502

A.R. Mosteo et al.

In future work, with the adoption of predictive signal models, parallelization of task execution may be achieved, without relinquishing the inherent network safeness of the C ONNECT T REE approach. The theoretical analysis may also be extended to bound actual performance in terms of the truly optimal performance for each metric. Acknowledgements. This work was partially supported by the Spanish project DPI200607928, the European project IST-1-045062-URUS-STP, and a Marie Curie International Reintegration Grant MIRG-CT-2006-044980 awarded to M. G. Lagoudakis.

References [1] Gerkey, B.P., Vaughan, R.T., Howard, A.: The player/stage project: Tools for multirobot and distributed sensor systems. In: Int. Conf. on Adv. Robotics, pp. 317–323 (2003) [2] Hwang, F.K., Richards, D.S., Winter, P.: The Steiner Tree Problem. Elsevier, Amsterdam (1992) [3] Kalra, N., Ferguson, D., Stentz, A.: A generalized framework for solving tightlycoupled multirobot planning problems. In: Int. Conf. on Robotics and Automation, p. 3359 (2007) [4] Lagoudakis, M.G., Markakis, E., Kempe, D., Keskinocak, P., Kleywegt, A., Koenig, S., Tovey, C., Meyerson, A., Jain, S.: Auction-based multi-robot routing. In: Robotics: Science and Systems Conf., Cambridge, USA (2005) [5] Marques, C., Cristoao, J., Lima, P., Frazao, J., Ribeiro, I., Ventura, R.: Raposa: Semiautonomous robot for rescue operations. In: Int. Conf. on Intell. Robots and Systems, pp. 3988–3993 (2006) doi:10.1109/IROS.2006.281836 [6] Mosteo, A.R., Montano, L., Lagoudakis, M.G.: Multi-robot routing under limited communication range. In: Int. Conf. on Robotics and Automation (2008) [7] Mostofi, Y.: Communication-aware motion planning in fading environments. In: Int. Conf. on Robotics and Automation (May 2008) [8] Pezeshkian, N., Nguyen, H.G., Burmeister, A.: Unmanned ground vehicle radio relay deployment system for non-line-of-sight operations. In: IASTED Int. Conf. on Robotics and Applications. DARPA/ITO, Germany (2007) [9] Qin, L., Kunz, T.: On-demand routing in MANETs: The impact of a realistic physical layer model. In: Pierre, S., Barbeau, M., Kranakis, E. (eds.) ADHOC-NOW 2003. LNCS, vol. 2865, pp. 37–48. Springer, Heidelberg (2003) [10] Rooker, M.N., Birk, A.: Communicative exploration with robot packs. LNCS (LNAI), pp. 267–278 (2006) [11] Sridhara, V., Bohacek, S.: Realistic propagation simulation of urban mesh networks. Computer Networks 51(12), 3392–3412 (2007)

Pipeless Batch Plant with Operating Robots for a Multiproduct Production System Satoshi Hoshino, Hiroya Seki, and Yuji Naka

Abstract. Chemical process industries need to be able to adapt to the fast changing market to meet consumer demands. A pipeless batch plant with multiple operating robots for a high-mix and low-volume production system which has more flexibility and agility is developed in this paper. For more efficient use of the plant resources, such as movable vessel robots, coupling robots, and production tasks, operational models for a vessel route planning, coupling robot operation dispatching, and task assignment are proposed as an integrated control methodology. Finally, the effectiveness of the operational models for the pipeless batch plant system is discussed and shown through the simulation experiment. Furthermore, a system bottleneck is analyzed for more effective plant system.

1 Introduction In chemical process industries, in terms of a reasonable and multiproduct production of high value-added chemical products in addition to the adaptation to the fast changing market, a pipeless batch plant has attracted attention over the past year, to meet consumer demands. A pipeless batch plant was derived from a batch plant. In the pipeless batch plant, materials are transported from one stage to another by movable vessels and a production process is executed at a number of fixed process stations. Therefore, in pipeless chemical batch plants, for instance, the following processes are conducted to a vessel or its contents: charging, blending, reaction (e.g., cooling and heating), discharging (for delivery), cleaning, etc. In a general batch plant, materials are transferred to reactor tanks (which are installed in the plant) through a pipe network and the production processes listed above are conducted into the tank as one batch; and then, final products are Satoshi Hoshino, Hiroya Seki, and Yuji Naka Tokyo Institute of Technology, 4259 Nagatsuta, Midori-ku, Yokohama, Kanagawa, 226-8503, Japan e-mail: {hosino,hseki,ynaka}@pse.res.titech.ac.jp

504

S. Hoshino et al.

produced. Regarding the pipeless batch plants, they are already in use to produce a number of chemical products, such as lubricants, adhesives, pharmaceuticals, paints, and inks. A favorable feature of the pipeless batch plant is that it is possible to transport materials or products themselves by using movable vessels instead of a pipe network. This means that a fixed reactor tank in a batch plant can move freely. Thus, the pipeless batch plant has a possibility to be more flexible and agile system and is able to prevent materials contamination when the materials or products are switched from batch to batch. Furthermore, each recipe for a production process is different from others; herewith, a multiproduct production in one plant is made possible. However, the efficient use of the plant resources must be guaranteed for the advantages described above. Therefore, in this paper, with the aim of improving the pipeless batch plant productivity (i.e., flexibility and agility), an integrated control methodology of operating robots and production tasks is proposed. This paper consists of the following sections: Sect. 2 where previous and related works are described; in Sect. 3, a challenges of this paper are mentioned; the detailed pipeless batch plant is explained in Sect. 4; operational models are proposed in Sect. 5; finally, through the simulation experiment in Sect. 6, the effectiveness of the operational models is discussed and a bottleneck that occurs in the system is analyzed.

2 Previous and Related Works A lot of previous studies that focused on pipeless batch plants have addressed a production scheduling problem. The scheduling problem has been formulated mainly with the use of the Mixed Integer Linear Programming, MILP ([1, 2]). By using the MILP method, Realff et al. have tackled a design problem with regard to a layout, such as the arrangement of process stations and the number of stations and movable vessels in addition to the scheduling problem [3]. However, a main weakness of the MILP approach is that, as the complexity of a plant increases, the scheduling problem becomes very hard to formulate properly [4]. Huang et al. have successfully optimized the scheduling, that is, minimization of the production makespan, as the Constraint Satisfaction Problem, CSP [5]. Moreover, they have proposed an integrated scheduling methodology in consideration of a behavior of movable vessels which is involved in different plant layouts [6]. These previous and related works, however, have been based on assumptions and had issues as follows: (1) fewer movable vessels; (2) constant transport time of a vessel between stations; (3) constant operation (process) time at a station; and (4) unconsidered coupling equipment to connect the vessel with other process equipment. Gonzalez et al. have investigated a discrete event production simulation on the basis of a scheduling solution which was obtained by using MILP method [7]. As a result, it has been shown that, as the case may be, an expected production volume according to the scheduling result is not achieved in the event that the vessel speed or the operation time at a process station fluctuates depending on circumstances. Additionally, vessels with huge volume (more than 10[m3]) have been used in the previous studies; hence, its mobility was row; it has been difficult to perform the

Pipeless Batch Plant with Operating Robots

505

vessels efficiently. In terms of plant safety, also, the moving of such a huge vessel has been viewed with suspicion.

3 Challenges In this paper, we (I) increase the number of movable vessels which have small volume with the use of the mobile robots, namely, the movable vessel robots in consideration of the plant safety; focus on (II) the effective movable vessel robot’s behavior, (III) the coupling operation between the vessel robot and other process equipment by the coupling robot, (IV) the continuous movement and operation of the vessel and coupling robots, and (V) the fluctuating vessel robot speed, i.e., the moving time and operation time between and at stations. Therefore, in order to improve the plant productivity, we tackle the following three challenges and develop operational models: 1. Route planning for the movable vessel robot 2. Operation dispatching to the coupling robot 3. Task assignment to the movable vessel robot

4 Pipeless Batch Plant From literatures [3, 6], we can see that three layouts, herringbone, circular, and linear, have been dealt with so far in a pipeless batch plant. In this paper, as the fourth and novel layout which includes these three layouts, the plant which has a circuit layout as shown in Fig. 1 is the objective. Since the plant consists of the main (oneway), passing (one-way), and intermediate (two-way) lanes, the movable vessel selects and changes a route depending on circumstances and is able to perform its task efficiently even if a traffic congestion or workload occurs on the main lane or at a process station. Furthermore, although a plant with a previous layout has the vessel collision and deadlock problems due to the bi-directional lanes that connect with each station, in the proposed layout, these problems are solved with relative ease by empowering the priority to a vessel robot moving on the intermediate lane because only the lane which connects the main and passing lanes is bi-directional. At each station, normally, certain chemical production processes, such as charging, blending, and reaction, as described in Sect. 1, are conducted. However, in this paper, we assume that all processes can be conducted at every station 1 ∼ 12 for a multipurpose plant. As for the discharging of a final product and vessel cleaning, these are the requisite processes, and therefore, the exclusive stations for them are provided, respectively. After an empty and clean movable vessel robot arrives at a process station (1 ∼ 12), the vessel has to be connected with a material tank for the material charge. In the blending and reaction processes, the vessel requires to be connected with a stirrer and reactivity meter. For this operation, a coupling robot works, on the inside of the main lane on which the vessel robots move, to connect the vessel with

506

S. Hoshino et al.

Process sta. 5

Process sta. 6

Process sta. 7

Process sta. 8

Lane for coupling robot 2 Process sta. 9

Coupling robot 2

Process sta. 4 Coupling robot 1

Passing lane (one-way)

Process sta. 2

Process sta. 1

Lane for coupling robot 1

Main lane (one-way)

Route for movable vessel robots

Process sta. 10

Coupling robot 3

Intersection (Lane joint)

Intermediate lane (two-way)

Coupling robot 4

Lane for coupling robot 3

Process sta. 3

Process sta. 11

Process sta. 12

Lane for coupling robot 4

Cleaning sta.

Discharging sta.

Fig. 1 Circuit Layout Structure of the Objective Pipeless Batch Plant System

these process equipment. A coupling robot moves on its own bidirectional lane only. These operating robots are fully autonomous and do have a map of the system; thus, are able to exchange information via distributed blackboards installed on the robots; finally, decide actions by themselves. This communication mechanism is called as a sign-board model [8]. As a massive parallel mechanism, the sign-board model provides immediate access to sign-boards of other robots at all times.

5 Operational Models for the Pipeless Batch Plant 5.1 Route Planning for Movable Vessel Robot In the objective plant system as shown in Fig. 1, the movable vessel robot has to transport a product as efficiently as possible while moving on three types of lanes adequately. For this purpose, we adopt the following three route planning approaches, and apply one of them to the vessel robots: (a) shortest-path routing; (b) dynamic routing looking ahead to one station; and (c) dynamic routing looking ahead to all the stations to a destination. Planning approach (a) finds the shortest route to a target for a vessel robot regardless of other vessel robots’ states. Planning approaches (b) and (c) are based on approach (a), but they find the second shortest route if there is/are vessel robot(s) at stations on the shortest route toward a target, according to circumstances. Approaches (b) and (c), in this case, make a detour to the target. At the start station in Fig. 2, when the vessel robot, MV1, plans a route to the goal station, it has to stop on the planned route due to the impeditive robots, such as

Pipeless Batch Plant with Operating Robots Fig. 2 Route Planning: a product transporting movable vessel robot (MV1) on the main lane is planning a route to the destination (goal) from the current position (start)

507

Route 1 MV1 Start station Re-planning

Route 2

MV2

Route 3 MV3

Stop Re-planning again Shortest route Goal station

MV2 and MV3, if the shortest-path routing (a) is applied. In order to respond to a changing situation, MV1 communicates a vessel robot on the next station, i.e., MV2 by applying planning approach (b) to the vessel robots; and then, MV1 plans a route again. At this time, if route candidate 1 from route candidates 1, 2, and 3 is selected, MV1 needs to replan a route to the passing lane again due to the obstacle, MV3. If MV1 selects route candidate 2, MV1 has to stop on the main lane before the station due to MV3. On the other hand, MV1 selects route candidate 3, by replanning a route with the use of planning approach (c), on the basis of the situation of all the stations to the destination. Communication for the route planning with the use of the approaches (b) and (c) is performed when the vessel robot passes through a joint of the main and intermediate lanes.

5.2 Operation Dispatching to Coupling Robot Coupling robots 1 ∼ 3 as shown in Fig. 1 work for the coupling operations of the movable vessel robots and process equipment at four stations, such as 1 ∼ 4, 5 ∼ 8, and 9 ∼ 12, respectively. When multiple vessel robots arrive at different stations (e.g., 1st, 2nd, 3rd, and 4th stations) at the same time, it is required to appropriately dispatch the operations to the coupling robot in order to improve the robot’s operating efficiency. Therefore, we focus on the coupling operation order; the coupling robot decides the next operation partner (vessel robot) and station on the basis of the following three dispatching rules: (a’) First-In First-Out, FIFO, (b’) neighborhood first, and (c’) minimization of the total moving distance. In the case as shown in Fig. 3, the coupling robot needs to reciprocate to right and left if rule (a’) is applied. To avoid these wastes of the movement and time, the coupling robot performs the operations smoothly in the following order: MV2 → MV3 → MV4 → MV1, by applying rule (b’). However, even rule (b’) incurs a wasted motion from MV4 to MV1. For this case, by applying rule (c’), the coupling robot is able to perform the operations most smoothly as follows: MV2 → MV1 → Fig. 3 Coupling Operation Dispatching: while a coupling robot is connecting the movable vessel robot, MV2, with a material tank at a station, vessel robots arrived at all other stations

Material tank

MV1 2nd arrival

MV2

Coupling operation

Arrival sequence: MV2 -> MV4 -> MV1 ->MV3 MV3

MV4

3rd arrival

1st arrival

508

S. Hoshino et al.

MV3 → MV4. Although this rule increases the computational cost as the number of stations increases, it obtains the minimum moving distance of the coupling robot between the stations by solving the combinatorial problem.

5.3 Task Assignment to Movable Vessel Robot After a movable vessel robot is washed at the cleaning station shown in Fig. 1, next production task is assigned to the robot. In a material handling system, dispatching rules to Automated Guided Vehicles (AGVs) for the material transport have been proposed [9]; additionally, these dispatching rules have been applied to a previous pipeless batch plant [10]. However, in these studies, heuristic rules to find only the next destination of the AGV have been taken into account. Compared to the general material handling system, in the pipeless batch plant system for the chemical products, it is impossible to address a movable vessel and transport product (or materials) in the vessel separately. Moreover, each product is produced on the basis of its own recipe. And all of the process for one product is carried out by the same vessel robot. Therefore, a suitable production task, which is a recipe that does not raise a workload in the system, needs to be selected and assigned to a vessel robot in each case. For this purpose, we introduce the following objective function; a task (Taskk ) is assigned to the vessel robot based on the function. The objective function denotes that a task with the lowest similarity to the execution state of the all tasks in the system is selected and assigned to the vessel robot. minimize ∑ ∑ Taskn,k (ExeTaskn − Taskn,k ), k

n

where, k is a reference task number and n represents a station number. ExeTaskn represents the total number of vessel robots which are being and going to be processed at stations n based on the production process recipes. As for Taskn,k , a binary variable, 0 or 1, is given whether a station n of the k-th reference task is a destination in the recipe for a product. In this regard, however, if a task is selected and assigned to a movable vessel robot by referring to all other unexecuted tasks on the basis of the objective function, tasks with fewer processes are gathered in the first half of the production activity, on the other hand, tasks which have many processes, that is, a heavy workload remains in Table 1 Total Time with 27 Operational Models

# of vessel robots 5

Average 10.02

10

7.78

15

7.61

Total operation time [h] Best 9.23 (b)(c’)(c”) 7.16 (b)(c’)(c”) 7.12 (c)(b’)(c”)

Worst 11.18 (a)(b’/c’)(b”) 8.62 (a)(a’)(b”) 8.27 (a)(c’)(b”)

Pipeless Batch Plant with Operating Robots

509

the second half. For this problem, we propose and apply the following three assignment policies: (a”) random; (b”) the whole task reference (∀k ∈ K); and (c”) a partial task reference (∀k ∈ K p ).

6 Simulation Experiment 6.1 Experimental Condition From Sect. 5.1, 5.2, and 5.3, in total, 3 × 3 × 3 = 27 operational models are applied in this simulation experiment. For this purpose, we developed a stochastic plant simulator with C++. The simulation results with 5, 10, and 15 movable vessel robots are compared and evaluated on the basis of the total operation time. The total number of tasks is 200, the process time at stations 1 ∼ 12 is determined 30 ∼ 80 [s] in a random manner. In the process recipe, a binary variable, 0 or 1, is given with a 1-in-3 chance or 2-in-3 chance, respectively, for each station. If a 1 is given to a station, the vessel robot goes to the station. At the discharging and cleaning stations, 10 ∼ 40 [s] and 20 ∼ 80 [s] are required in a random manner for the production processes. With regard to assignment policy (c”) described in Sect. 5.3, the partial task reference range is 10 tasks (i.e., K p = 10).

6.2 Effectiveness of the Operational Models Table 1 shows the simulation result for the number of movable vessel robots, 5, 10, and 15, respectively. The averaged total operation time with 27 operational models is described in column “average.” In columns of “best” and “worst,” the shortest and longest operation time with operational models are described. Under the best and worst operation time, the combination of the applied operational models is shown. From this result, regardless of the number of movable vessel robots, the best case resulted in a reduction of 0.5 [h] and more than 1.0 ∼ 2.0 [h] compared to the average and worst total operation time. Furthermore, the plant system, in which the combinatorial operational models, such as the dynamic route planning, distancebased dispatching, and partial task reference-based assignment are applied, resulted in the highest productivity. The system operation time obtained with the use of the operational models, such as the route planning, operation dispatching, and task assignment, are shown in Fig. 4. In order to compare and evaluate the impacts of each of three models, (a) ∼ (c), (a’) ∼ (c’), and (a”) ∼ (c”) on the plant system efficiency, for example, in Fig. 4(a), the averaged operation time obtained through the simulations with fixed route planning (a), (b), or (c) and other nine combinatorial operational models ((a’) ∼ (c’) × (a”) ∼ (c”)) are shown. From these results, we can see that the route planning in Fig. 4(a) and then the task assignment in Fig. 4(c) have the most and more impacts on the system operation time. Here, note that operational models with the task assignment policy (b”)

510

S. Hoshino et al.

12

12 : Shortest-path routing (no route replaning)

10

: Dynamic routing looking ahead (to the next statation) : Dynamic routing looking ahead (to the target station)

9 8 7 6

: FIFO-based 11 Operation Time [h]

Operation Time [h]

11

: Neighborhood-based : Total moving distance-based

10 9 8 7

5

10 Number of Movable Vessel Robots

(a) Route planning

15

6

5

10 Number of Movable Vessel Robots

15

(b) Operation dispatching

12 : Random-based Operation Time [h]

11

: Whole task reference-based : Partial task reference-based

10 9 8 7 6

5

10 Number of Movable Vessel Robots

15

(c) Task assignment Fig. 4 Comparison and Evaluation of the Impact on the System Operation Time

resulted in the most inefficient system (see Fig. 4(c)) due to the workload described in Sect. 5.3. On the other hand, the operation dispatching rules applied to the coupling robots did not affect the system operation time materially as shown in the result of Fig. 4(b). This is because only two or four stations are handled by one coupling robot and their interval is relatively short (a few meters in the plant system). However, as can be seen from the result of Table 1, these rules have some benefit by combining with other operational models.

6.3 Bottleneck Analysis for More Effective Plant System In Fig. 5, as the robots utilization, the sojourn time ratio of the movable vessel robots (5, 10, and 15) at each station and operation time ratio of the coupling robots (1 ∼ 4) are shown on the basis of the “best” result in Table 1. From the results of Fig. 5(a), Fig. 5(c), and Fig. 5(e), it is noticeable that the sojourn time ratio at the discharging and cleaning stations increases as the number of movable vessel robots increases from 5 to 10 and 15. It is for this reason that, while the system throughput at the stations 1 ∼ 12 was improved by using the efficient operational models, the vessel robots often arrived at the discharging and cleaning stations where no passing lane is provided; eventually, these stations on the main lane became the system bottleneck. On the other hand, from the comparison of the results shown in Fig. 5(b), Fig. 5(d), and Fig. 5(f), we can see that each coupling

Pipeless Batch Plant with Operating Robots

Discharging & Cleaning stations 28 [%]

Stations 1, 2, 3, 4 23 [%]

Coupling robot 4 26 [%]

511

Coupling robot 1 24 [%]

Stations 1, 2, 3, 4 Discharging &

19 [%]

Cleaning stations 42 [%]

Stations 5, 6, 7, 8 Stations 9, 10, 11, 12

23 [%]

Coupling robot 3 25 [%]

25 [%]

18 [%] Stations 9, 10, 11, 12

26 [%]

21 [%]

(a) Sojourn ratio (5)

Coupling robot 4 28 [%]

(b) Operating ratio (5)

23 [%]

Coupling robot 2 24 [%]

25 [%]

(d) Operating ratio (10)

(c) Sojourn ratio (10)

Stations 1, 2, 3, 4

Coupling robot 1 Discharging & Cleaning stations

Coupling robot 3

Stations 5, 6, 7, 8

Coupling robot 2

48 [%]

17 [%]

Coupling robot 4 28 [%]

Coupling robot 1 23 [%]

Stations 5, 6, 7, 8 18 [%]

Stations 9, 10, 11, 12 17 [%]

(e) Sojourn ratio (15)

Coupling robot 3 25 [%]

Coupling robot 2 24 [%]

(f) Operating ratio (15)

Fig. 5 Robots Utilization in the Best Plant System (# of Movable Vessel Robots)

robot operates almost evenly. That is to say, in spite of all coupling robots operate almost evenly and completely, the system bottleneck has occurred at two stations. The authors, so far, have shown the effectiveness a robots’ behavior in which an idling robot supports another robot’s operation reactively for a locally occurring heavy workload which changes with time [11]. Hence, although the coupling robot was assumed to work on its own lane only in this paper (see Sect. 4), we believe that it is possible to improve the system operating efficiency by dispatching other tasks at other process stations, which are handled by another coupling robot, to the idling robot. In other words, the coupling robots move to other lanes according to the situation for workload balancing. By doing this, the bottleneck would disappear.

7 Conclusion In this paper, with the aim of improving a pipeless batch plant productivity with multiple operating robots, an integrated control methodology for the plant system, such as the route planning for the movable vessel robots, operation dispatching to the coupling robots, and task assignment to the vessel robot, was proposed. From the results of the simulation experiment, we showed the effectiveness of the operational models for the pipeless batch plant system. Furthermore, for more effective plant system, a bottleneck which occurred in the system was analyzed on the basis of the robots utilization. This problem will be addressed in our future work.

512

S. Hoshino et al.

Acknowledgements. This research was partially supported by the Japanese Ministry of Education, Science, Sports, and Culture, Grant-in-Aid for Young Scientists (B), 19760167, 2007– 2008.

References 1. Bok, J.-W., et al.: Continuous-Time Modeling for Short-Term Scheduling of Multipurpose Pipeless Plants. Industrial & Engineering Chemistry Research 37(9), 3652–3659 (1998) 2. Lee, K.-H., et al.: Continuous Time Formulation of Short-Term Scheduling for Pipeless Batch Plants. Journal of Chemical Engineering of Japan 34(10), 1267–1278 (2001) 3. Realff, M.J., et al.: Simultaneous Design, Layout and Scheduling of Pipeless Batch Plants. Computers and Chemical Engineering 20(6), 869–883 (1996) 4. Huang, W., et al.: Scheduling of Batch Plants: Constraint-based Approach and Performance Investigation. International Journal of Production Economics 105(2), 425–444 (2007) 5. Huang, W., et al.: Scheduling of Pipeless Batch Plants Using Constraint Satisfaction Techniques. Computers and Chemical Engineering 24(2), 377–383 (2000) 6. Huang, W., et al.: Integrating Routing and Scheduling for Pipeless Plants in Different Layouts. Computers and Chemical Engineering 29(5), 1069–1081 (2005) 7. Gonzalez, R., et al.: Operation of Pipeless Batch Plants - I. MILP Schedules. Computers and Chemical Engineering 22(7), 841–855 (1998) 8. Wang, J.: On Sign-board Based Inter-robot Communication in Distributed Robotic Systems. In: IEEE International Conference on Robotics and Automation, pp. 1045–1050 (1994) 9. Egbelu, P.J., et al.: Characterization of Automatic Guided Vehicle Dispatching Rules. International Journal of Production Research 22(3), 359–374 (1984) 10. Gonzalez, R., et al.: Operation of Pipeless Batch Plants – II. Vessel Dispatch Rules. Computers and Chemical Engineering 22(7), 857–866 (1998) 11. Hoshino, S., et al.: Development of a Flexible and Agile Multi-Robot Manufacturing System. In: 17th IFAC World Congress, pp. 15786–15791 (2008)

Control Methodology of Transfer Crane with Look-Ahead Rule in Harbor Transportation System Hisato Hino, Satoshi Hoshino, Tomoharu Fujisawa, Shigehisa Maruyama, and Jun Ota

Abstract. Currently, container ships move cargo with minimal participation from inland chassis trucks. However, there is slack time between the departure of container ships and the completion of cargo handling of container ships without the participation of inland chassis trucks; therefore, inland chassis trucks can be used to move cargo without delaying the departure time. In this paper, we propose a solution involving the control algorithms of transfer cranes (TC). The TCs work according to the heuristic rules with look-ahead to forecast the arrival times of yard and inland chassis trucks. Simulation results show that the waiting time of inland chassis trucks can be reduced to meet the departure times of container ships.

1 Introduction 1.1 Research Background The globalization of transportation in recent years has spurred enormous growth in the size of container ships and the number of managed containers. Container Hisato Hino Department of Precision Engineering, Graduate School of Engineering, The University of Tokyo, Bunkyo-ku, Tokyo 113-8656, Japan e-mail: [email protected] Satoshi Hoshino Chemical Resources Laboratory, Tokyo Institute of Technology, Yokohama, Kanagawa 226-8503, Japan Tomoharu Fujisawa Ministry of Land, Infrastructure, Transport and Tourism, Chiyoda-ku, Tokyo 100-8918, Japan Shigehisa Maruyama Japan Association of Cargo-handling Machinery Systems, Minato-ku, Tokyo 105-0003, Japan Jun Ota Department of Precision Engineering, Graduate School of Engineering, The University of Tokyo, Bunkyo-ku, Tokyo 113-8656, Japan

514

H. Hino et al.

terminals have smaller area in Japan than in other countries due to limited land. In addition, the number of containers transported between terminals and inland is considerable in Japan. Therefore, the increase in the number of managed containers has a direct influence on the operational efficiency of terminals. In this respect, the construction of an efficient container terminal management system is indispensable. First of all, solutions to these types of problems are essential before an efficient container terminal management system can be constructed.

1.2 Container Terminal System A container terminal system consists of working machines, a container yard area, entrance and exit gates. Working machines include gantry cranes, transfer cranes (TCs), and vehicles (yard chassis and land-side chassis). A container yard area stores containers which are stacked in storage lanes. A general view is shown in Fig. 1. Container terminal operations are classified into sea-side operations and landside operations. Sea-side operation involve the transportation of containers between ships and a terminal. Land-side operations include the transportation between a terminal and an inland destination (e.g. a factory). These two operations are further classified into move-in/-out operations, in which containers are brought into a terminal or carried out of a terminal. In the sea-side operation, a gantry crane loads/unloads a container to/from the ship, and a yard chassis transports the container between the ship and the yard area. A gantry crane is mounted on a quay. A TC, which is a mobile crane, unloads the container and stores it at the storage position in the storage lane. In the land-side operation, a land-side chassis move a container between the yard area and inland. After a TC loads/unloads the container, the land chassis passes an exit gate and leaves the terminal. The container terminal service starts when container ships arrive at the terminal. Simultaneously, land-side chassis come to the terminal one after another for the land-side operation. The departure time of each ship cannot be changed because it is determined in advance based on the number of managed containers or their next destination (here, the time from the arrival of a ship to the departure is defined as the scheduled sea-side operation time). Therefore, it is necessary to keep the constraint Yard chassis Land-side chassis

Storage lane Yard area

Entrance gates for movein chassis Entrance gates for moveout chassis

Exit gates for moveout chassis Exit gates for movein chassis

Fig. 1 General view of container terminal

Gantry crane TC (Transfer Crane)

Control Methodology of Transfer Crane with Look-Ahead Rule

515

of the scheduled sea-side operation time, according to which the sea-side operation must be finished. However, giving excessive priority on sea-side operations results in long waits for the land-side chassis at the entrance gates. From the viewpoint of the quality of container terminal service, it is advisable that the waiting time of the land-side chassis is as short as possible. Hence, the following two demands are placed on container terminal systems: • To keep the constraint of the scheduled sea-side operation time, which is most important. • To reduce the waiting time of the land-side chassis as much as possible. As mentioned, the volume of land-side chassis is greater at Japanese terminals than in other countries. Under these existing circumstance, terminal authorities give high priority to sea-side operations to keep the constraint of scheduled sea-side operation time. As a consequence, large numbers of land-side chassis waiting at entrance gates cause traffic congestion around terminals. In such situation, the need to reduce the waiting time of land-side chassis is not satisfied. On the other hand, as a result of giving priority to sea-side operation, there tends to be a slack time on the end of scheduled sea-side operation time which results to imbalance in the amount of work done by sea-side and land-side operations. Thus, a solution that satisfies both demands for container terminal system is necessary.

1.3 Related Studies A considerable number of studies related to terminal systems have been conducted: the layout of a terminal [1][2], the optimum number of machines within a terminal [1][3], the behavior of working machines [4][5], the task priority [6][7][8], the container storage allocation [8][9]. In [11], simulation models of container terminal are proposed. However, little attention has been given to land-side operations. [6] and [12] discuss land-side operations, but, in [6], the order of tasks is given beforehand, without considering the land-side chassis waiting at entrance gates. In [12], control of the behavior of chassis is the main purpose, and the two demands for terminal system are left out of consideration.

1.4 Research Purpose The purpose of this paper is to propose a solution to reduce the waiting time of land-side chassis as much as possible while keeping the constraint of the scheduled sea-side operation time. In this case, it is difficult to achieve the required balance between sea-side and land-side operations. Well-balanced terminal operations are achieved by controlling the machines used to perform terminal operations. Among working machines, the TCs, which manage yard and land-side chassis, are the most important. Thus, in this study, we attempt to satisfy both demands of an efficient terminal system by controlling the behavior of TCs. To control TCs, it is very important to realize well-balanced operations. Therefore, we propose control rules of

516

H. Hino et al.

TCs that can maintain the balance. In the proposed rules, TCs determine to manage yard chassis or land-side chassis using the information obtained by looking-ahead, that is, forecasting the arrival time of chassis to the lane.

2 Problem Settings Terminal environment settings are based on a daily operational flow. At the start of container terminal service, a container ship arrives, and sea-side operations begin. Simultaneously, land-side chassis arrive to perform land-side operations. The arrivals of land-side chassis follow the specific distribution. The distribution roughly indicates a tendency; however, the correct arrival time cannot be determined.

2.1 Behavior Rule of TCs Each storage lane has a buffer at the entrance of the lane which can hold up to two yard chassis and two land-side chassis. Fig. 2 shows two yard chassis and land-side chassis waiting at the entrance of a storage lane. A land-side chassis arrives at an entrance gate and waits there. Then, if the number of land-side chassis waiting at the entrance of storage lane to which the land-side chassis will go (we define this lane as the target storage lane) is below 2, the landside chassis passes the entrance gate, goes to the buffer of target storage lane, and waits there. Otherwise, the land-side chassis continues to wait at the entrance. One chassis cannot overtake another chassis. When a yard or a land-side chassis proceeds to the yard area, it informs the TC in charge of the target storage lane of its arrival time at the target storage lane as well as of the storage position of the container. The TC determines which chassis to manage next on the basis of the available information. The selected chassis enters the storage lane and proceeds to the storage position. The TC also goes to the storage position after the determination. The other chassis continue to wait at the buffer until they are selected by the TC. When both the chassis and the TC reach the storage position, the container is loaded/unloaded. The most important aspect in this situation is that which chassis the TC chooses based on the situation. The balance between the seaside and land-side operations must be maintained to satisfy the two demands of the terminal system. Poor choices by the TC disturb the balance and impede the efficiency of the terminal operations. Yard-chassis

Land-side chassis

TC

Fig. 2 Lines of waiting chassis at the buffer of storage lane

Control Methodology of Transfer Crane with Look-Ahead Rule

517

3 Model Framework 3.1 The Presently-Used Method As noted, the sea-side operation is given priority to keep the constraint of the scheduled sea-side operation time. Namely, at present, TCs choose the next chassis according to the sea-side priority rule, in which if a yard chassis has already arrived at the target storage lane, a yard chassis is chosen. Otherwise, if the choice of land-side chassis does not influence the sea-side operation, the TC will choose the land-side chassis, that is, if the yard chassis does not arrive before the completion of the operation of the land-side chassis, the land-side chassis is chosen.

3.2 The Flow of Choice of Chassis To satisfy the demands of the terminal system, the TC has to choose the chassis appropriately. However, it is difficult to decide which chassis to choose in the long term because when arrival times at the terminal and storage positions of land-side chassis are unclear. Therefore, we propose dispatching rule-based algorithm of behavior of TCs. The TC chooses the chassis in accordance with the flowchart shown in Fig. 3. First, the TC checks the progress of the sea-side operation to maintain the operational schedule on the sea-side. We consider the present state of progress and the target state of progress (the sea-side operation will finish precisely in the scheduled sea-side operation time) as indices of the state of progress. In brief, we define the target working efficiency and the present working efficiency of sea-side operation as follows. The target working efficiency is given as the target working efficiency =

ntotal , tscheduled

(1)

where ntotal is the total number of containers in the sea-side operation and tscheduled is the scheduled sea-side operation time. The present working efficiency is given as the present working efficiency =

n present , t present

(2)

Start of choice

N

(a)

Fig. 3 Flow chart of choice

Adopt the sea-side priority rule

Is the target efficiency of seaside operation is smaller than the present efficiency? Y

Adopt one of the proposed rules

End of choice

(b)

518

H. Hino et al.

where n present is the number of containers that have been managed in the sea-side operation at t present , which is the time from the start of the sea-side operation. (1) is the average pace of the sea-side operation to finish precisely in the scheduled sea-side operation time. (2) indicates the actual state of progress of the sea-side operation at t present . The TC chooses the chassis by comparing these two indices. If the present working efficiency does not amount to the target working efficiency ((a) in Fig. 3), TCs determine which chassis to manage according to the sea-side priority rule to keep the constraint of scheduled sea-side operation time.

3.3 The Proposed Rules On the other hand, if the present working efficiency is higher than the target working efficiency, that is, the sea-side operation has time to spare, the TC chooses a chassis according to one of the rules proposed after this ((b) in Fig. 3). The proposed rules can reduce the waiting time of the land-side chassis. If the sea-side operation has time to spare, the TC chooses chassis according to the proposed rules. When the sea-side operation has time to spare, we propose following the land-side priority rule as the simplest rule. In this rule, if a land-side chassis has already arrived at the target storage lane, the land-side chassis is chosen. If the choice of yard chassis does not influence the land-side operation, the TC will choose the yard chassis. The land-side priority rule can improve the terminal efficiency, but there is still room for improvement. In this rule, the TC chooses a chassis considering only those chassis which are candidates for the next management. However, in the look-ahead rule, the TC chooses chassis more effectively by considering not only those chassis which are candidates for the next management but also those which will be managed after them. In this rule, the TC finds all feasible management orders of the chassis in the range of information that the TC can obtain at that point and then calculates the total operation time, which includes the time costs to manage the chassis according to each order. The next chassis is determined on the basis of the order with the shortest total operation time. The TC determines the chassis to manage next again Yard-chassis Land-side chassis

(13:10)

(d) (c)

(13:06)

TC

(a) (13:05)

(b) (13:07)

(a) (c) (b)

(d)

Fig. 4 Example of the choice of TC

containers

Control Methodology of Transfer Crane with Look-Ahead Rule

519

when the TC obtains new information; for instance, a task is assigned to a yard chassis or a land-side chassis passes an entrance gate. In this rule the TC considers yard chassis that comes to the lane next and land-side chassis that have already passed entrance gates, because choise of after these chassis influence other lanes one another very complicatedly. Now, we assume that there are two yard chassis and two land-side chassis waiting in line at the buffer shown in Fig. 4 as an example. The symbols ”(a),” ”(b),” ”(c),” and ”(d)” represent each chassis, and their placements in the storage lane in Fig. 4 represent the storage position of each chassis. The arrival time of the chassis is shown beside each one. The TC finds the best management order under the constraint that (a) is to be managed earlier than (c), and (b) earlier than (d). If the landside priority rule is adopted, the TC chooses (b). If the look-ahead rule is adopted, the TC decides to manage (a) next according to the best management order which is (a)-(c)-(b)-(d). It is noteworthy that the TC finds the best management order to determine the next chassis and does not manage the chassis according to the order. After the completion of management, the TC finds the best order and chooses another chassis.

4 Simulation 4.1 Simulation Settings The proposed rules are expected to adjust the state of progress of the sea-side operation according to the scheduled sea-side operation time. In the simulation, the scheduled sea-side operation time is set to be from 30 minutes to 120 minutes longer than than the actual sea-side operation time with the presently-used method (9.25 hours). Table 1 Simulation Settings Number of entrance gates Number of exit gates Container storage position Number of yard chassis Number of gantry cranes Number of TCs Number of containers unloaded from a container ship Number of containers loaded onto a container ship Number of move-out containers (=Number of move-out land-side chassis) Number of move-in containers (=Number of move-in land-side chassis) Container storage position Surplus time Number of storage lanes

5 (for move-in/-out) 1 (for move-in/-out) fixed in advance (random) 12 3 1 (each lane) 300 300 500 500 change randomly each time 30, 60, 90, 120 [minutes] 10

520 11.5 sea side operation completion time (hours)

Fig. 5 Sea-side operation completion time

H. Hino et al.

11 10.5

land-side priority look-ahead scheduled time the presently-used method

10 9.5 9 8.5

Fig. 6 Average waiting time of land-side chassis

average waiting time of land side chassis (hours)

30

60 90 surplus time (min.)

120

3 2.5 2 1.5 1 0.5

land-side priority look-ahead the presently-used method

0 30

60

90

120

surplus time (min.)

This time to spare is defined as surplus time. Simulation settings are shown in Table 1. Storage positions of containers are allocated randomly to evaluate the effect of the control of TCs as clear as possible. We ran five simulation in which storage positions of containers were randomly changed, and the results were averaged.

4.2 Results and Evaluation Figure 5 and Figure 6 show the sea-side operation completion time and the average waiting time of the land-side chassis at entrance gates using the proposed rules. Figure 7 shows the waiting time of each move-out land-side chassis when the TC adopts the presently-used method and the look-ahead rule, and the surplus time is changed from 30 minutes to 120 minutes. The surplus time does not affect the results in which the sea-side priority rule is adopted as shown in Fig. 5 to Fig. 7, since the sea-side priority rule always gives priority on the sea-side operation and does not consider whether the sea-side operation has time to spare or not. In Fig. 7, the waiting time of each land-side chassis increases with time because the land-side chassis has to wait in line, and the time from arrival at the terminal to the passage through the entrance gates gets longer. As time passes, the waiting time becomes more uneven, which seems to be caused by the difference in the line length at the entrance gates. In Fig. 6, using the look-ahead rule, when the surplus time is 30 minutes (the scheduled sea-side operation time is 9.75 hours), the average waiting time is reduced by about 27%, and when it is 120 minutes (11.25 hours), it is reduced by about 56% compared to the presently-used method. As Fig. 6 indicates, the average waiting time of the land-side chassis decreases with increasing scheduled sea-side

Fig. 7 Comparison of waiting time of each chassis between the presently-used method and the proposed (look-ahead) method

waiting time of each chassis (hours)

Control Methodology of Transfer Crane with Look-Ahead Rule

521

5

30 minutes surplus time 60 minutes surplus time 90 minutes surplus time 120 minutes surplus time the presently-used method

4.5 4 3.5 3 2.5 2 1.5 1 0.5 0 0

100 200 300 400 number of chassis passed entrance gates

500

operation time. On the other hand, the sea-side operation completion time increases with increasing scheduled sea-side operation time, as shown in Fig. 5. These two figures mean that the TC can reduce the average waiting time by managing land-side chassis appropriately according to the scheduled sea-side operation time.

5 Conclusion In this paper, we proposed a method to satisfy two demands of a container terminal system, which are to keep the constraint of the scheduled sea-side operation time and to reduce the waiting time of land-side chassis as much as possible by controlling the behavior of TCs according to the state of progress of the sea-side operation. As a result, the average waiting time of a land-chassis is reduced by at most about 56% compared to the presently-used method while keeping the constraint of the scheduled sea-side operation time.

References 1. Liu, C.-I., Jula, H., Vukadinovic, K., Ioannou, P.: Automated guided vehicle system for two container yard layouts. Transportation Research Part C: Emerging Technologies 12(5), 349–368 (2004) 2. Taghaboni-Dutta, F., Tanchoco, J.M.A.: Comparison of dynamic routing techniques for automated guided vehicle system. International Journal of Production Research 33(10), 2653–2669 (1995) 3. Hoshino, S., Ota, J., Shinozaki, A., Hashimoto, H.: Hybrid design methodology and cost-effectiveness evaluation of AGV transportation systems. IEEE Transactions on Automation Science and Engineering 4(3), 360–372 (2007) 4. Adulyasak, Y., Lohatepanont, M.: Reducing truckload transportation costs through continuous move optimization. In: The 2nd International Conference on Transportation Logistics (2007) 5. Zhang, C., Wan, Y.W., Liu, J., Linn, R.J.: Dynamic crane deployment in container storage yards. Transportation Research, Part B (Methodological) 36(6), 537–555 (2002) 6. Ng, W.C., Mak, K.L.: Yard crane scheduling in port container terminals. Applied Mathematical Modeling 29(3), 263–276 (2005) 7. Lee, D., Wang, H., Miao, L.: A genetic algorithm for quay crane scheduling with handling priority in port container terminals. In: The 2nd International Conference on Transportation Logistics (2007)

522

H. Hino et al.

8. Grunow, M., Gunther, H.-O., Lehmann, M.: Dispatching multiload AGVs in highly automated seaport container terminals. OR Spectrum 26(2), 211–235 (2004) 9. Nishimura, E., Imai, A., Papadimitriou, S.: Storage allocation problem for transshipment containers at the container terminal for mega-containerships. In: The 2nd International Conference on Transportation Logistics (2007) 10. Zhao, S., Liu, W., Zhang, X.: A method for real-time storage space allocation of export containers in container terminals. In: The 2nd International Conference on Transportation Logistics (2007) 11. Bielli, M., Boulmakoul, A., Rida, M.: Object oriented model for container terminal distributed simulation. European Journal of Operational Research 175(3), 1731–1751 (2006) 12. Zhang, J., Ioannou, P.A.: Automated container transport system between inland port and terminals. ACM Transactions on Modeling and Computer Simulation 16(2), 95–118 (2006)

A Novel Marsupial Robot Society: Towards Long-Term Autonomy Marek Matusiak, Janne Paanajärvi, Pekka Appelqvist, Mikko Elomaa, Mika Vainio, Tomi Ylikorpi, and Aarne Halme 1

Abstract. Within this paper a novel marsupial multi-robot system intended for long-term operation, the Marsubot Society, is introduced. The Marsubot Society is a platform for robotics and multi-robot system algorithm development. In a system that needs to operate autonomously for extended periods of time, energy is of the essence. The challenges that lie in the energy economy are how to distribute and access energy and what the system can do with the resources. The multi-robot system’s capabilities and limitations are discussed. Results of preliminary tests are presented and ideas for future work introduced.

1 Introduction This paper presents the Marsubot Society, a novel multi-robot marsupial system for long-term autonomy. It consists of two different types of robots: Marsubots (marsupial society unit robots) and the marsupial mother unit, the Motherbot. Long-term autonomy poses many challenges for the operation of multi-robot systems. The Marsubot Society tackles these challenges by providing different methods of energy transfer and versatile robots. It also aims to be a platform for diverse multi-robot algorithm research. Results from this development will be detailed in future publications. Other key ideas behind the system include modularity of the components, flexibility of the system to upgrades (new sensors etc.) and sufficient processing power to run image processing and filtering algorithms. An operational case in designing the system has been patrolling and monitoring a building for intruders, tracking targets and asking for identification when necessary. Marek Matusiak, Janne Paanajärvi, Pekka Appelqvist, Mikko Elomaa, Mika Vainio, Tomi Ylikorpi, and Aarne Halme1 Finnish Centre of Excellence in Generic Intelligent Machines Research, Department of Automation and Systems Technology, Helsinki University of Technology, Finland e-mail: [email protected]

524

M. Matusiak et al.

The ability to transition from floor to floor has also been deemed to be essential – for that reason and for recharging Marsubots the system has a marsupial mother unit which can transport smaller robots to otherwise inaccessible places. Chapter two presents the members of the Marsubot Society. Chapter three details the systems energy economy and preliminary methods and test results of docking. Chapter four concludes the paper and presents ideas for future research. The authors’ research group has a long tradition in multi-robot research since 1993. Back then, the ideas of the nature-inspired robot society architecture along with potential applications were presented [1]. The first implementation of the concept was the SUBMAR robot system [2][3][4] targeted to distributed autonomous perception and task execution in the internal, three-dimensional on-line monitoring of various liquid flow-through processes. The SUBMAR system was followed by the introduction and realization of another novel underwater multirobot system, SWARM [5]. In this case, the application was related to environmental monitoring of the shallow water sea areas, with a two-week autonomous mission capability. In the robot society systems mentioned in [1-5] there has been a strong biological analogy. A biological inspiration for the Marsubots’ energy exchange can be found from the liquid food exchange by regurgitation of certain ant species (e.g. workers of the species Formica sanguinea) [6]. One of the earliest marsupial robot systems was the team of floor characterization robots MACS and RACS [7]. The bigger MACS robot was used to explore areas it could reach and to carry and deploy the smaller RACS robot near limited access areas as designated by the operator and as a repeater for improving radio communications. The term ‘marsupial robot’ was introduced in [8]. The system presented featured two robots: Silver Bullet and Bujold. The car-like Silver Bullet functioned as the dispensing agent or mother and the micro-rover Bujold as the passenger agent or daughter. However, Bujold did not have any onboard processing ability or power; both of these were handled by Silver Bullet (power via tether) [9]. The system was used in urban search and rescue domain. A second system was also built by Murphy et al [9][10], consisting of an iRobot ATRV mother and three iRobot Urban daughters. In [11] a marsupial robotic team was presented which featured small robots were deployed from their mother by means of a launcher, which was used to propel them up to 30 m. The small ‘scout’ robots have also the distinctive capability to jump over small vertical obstacles. In [12] a conceptual model of a mobile docking station is presented which would also be able to recharge the scout robots. Energy exchange between mobile robots by exchanging batteries has been presented in [13]. The robots, called the CISSbots, moved on a grid on the floor which they use for localization in addition to odometry. Docking and the battery exchange were handled by arranging a meeting point on the grid and using infrared local communication. A conceptual design to add small “microbot” daughters to the system was also introduced, with the CISSbot functioning as the marsupial mother.

A Novel Marsupial Robot Society: Towards Long-Term Autonomy

525

2 System Description The presented heterogeneous multi-robot system consists of two different types of robots: Marsubots and the Motherbot. See Fig.1 below.

Fig. 1 Left: three Marsubots on the foreground, the Motherbot prototype in the background. Right: Marsubot entering the Motherbot

This robot society will serve as a research and test platform for different multirobot algorithms e.g. for localization, Multi-SLAM, planning, image processing, task allocation, resource allocation and for control architecture design and testing. The results of these experiments will be explored in future publications. See Fig. 2 below for the schematics of the robots’ hardware configuration and different data and power buses used. Marsubots are small (length 24 cm, width 17 cm, height 20 cm) holonomic robots designed for indoor operation. Marsubot’s main sensors are a camera and a laser scanner. The Motherbot is the marsupial mother unit and thus the largest of the robots. It is capable of carrying three Marsubots while recharging their batteries. While Marsubots are designed to be small enough to be transported, they also have enough processing power and energy to operate autonomously in an indoor environment. Ten Marsubots will be built in total.

+5V

+5V

I2C

Ultrasonic Rangefinders

USB Webcam

RS232

Ultrasonic Rangefinders

WLAN Radio

Processor 1: Ampro Readyboard Pentium M 1.4GHz

+5V

+5V Hokuyo URG Ladar +5V

+12V

RS232

+12V H-Bridge

Ctrl

Processor 1: Gumstix Verdex ARM9

RS232

+5V CF Card

Encoders

+5V

+5V

+24V

Processor 2: PhyCore 167CR 16-bit uC

RS232

RS232

Ctrl

Recharging point 2: Rear

Powerboard

+5V

Processor 3: Atmel AVR 8-bit uC

Ctrl

Ctrl Motors

Ctrl, Data

Encoders +5V Processor 2: PhyCore 167CR 16-bit uC

Data

Recharging point 1: Beak

WLAN Radio

+24V H-Bridge

+5V

Ctrl Motors

Hokuyo URG Ladar

Webcams

USB

IR Stair Detector

Data

Battery Recharger 1 +5V Battery Recharger 2

Ctrl

+24V

+12V Powerboard

Battery Recharger 3 +5V

Recharging point 3: Bottom

Fig. 2 Hardware schematics for the Marsubots (Fig. 2a, left) and the Motherbot (Fig. 2b, right)

526

M. Matusiak et al.

The real world case for the Marsubot Society involves patrolling the building where our laboratory resides in and tracking and identifying targets (e.g. by means of camera and possibly RFID). The long-term operating goal of the system is for it to be able to patrol and survey our laboratory overnight and for extended periods like over the weekend. The Motherbot is able to drive up and down the stairs to relocate the Marsubots to different floors.

2.1 Marsubot Marsubot is a three wheeled mobile robot with two DC-motors and two processors. Its sensors include a web camera and a laser scanner. There are optional design plans to implement ultrasonic sensors for obstacle avoidance and a downward facing IR-sensor for detecting stairs. See Fig. 3 below and Fig. 2a above for hardware references.

Fig. 3 Marsubot

The two front wheels are 15 cm in diameter and coated with soft rubber to help the robot to overcome obstacles such as doorsteps. The third wheel, a castor, is used to stabilize the robot. Two separate computers are used in the design, a 16-bit microcontroller (Fig. 2a, Processor 2) for real time controls and sensor interfacing, and an ARM9based Linux based computer operating at 400 or 600 MHz (Fig. 2a, Processor 1) for high-level controls, camera interfacing and communications. The power electronics enable the turning off of all other actuators and sensors other than the microcontroller, which is always left on to monitor the current energy state and possible recharging connections. However, for sensing the environment and communications Processor 1 is also needed. Marsubot’s energy capabilities are described in chapters 3.1 and 3.2.

A Novel Marsupial Robot Society: Towards Long-Term Autonomy

527

2.2 Motherbot The second type of robot in the multi-robot system is the Motherbot. It is designed for transporting the smaller Marsubots as well as recharging them. It will also be a natural communications and control hub, with more processing capacity and better sensors. Motherbot is run with two motors and is track driven. The tracks enable it to climb stairs making it possible for the team to cover and be spread out in multiple floors of a building at the same time. Marsubots will be housed in its cargo hold between its tracks. The cargo hold is elevated from the ground and Marsubots will reach it using ramps which can be lowered for this purpose. Inside the hold there are voltage rails for three Marsubots. See Fig. 2b for the Motherbots subsystem schematics and processor references. The Motherbot’s main sensor, a laser scanner, will be complemented with cameras and ultrasonic rangefinders. Three different processors are used, a Pentium Mobile based Linux-box (Processor 1) for high level controls and communications, a similar 16-bit microcontroller (Processor 2) to the one used in the Marsubots for motor and sensor interfacing and real-time controls and an 8-bit microcontroller (Processor 3) for handling the three different battery rechargers connected to the rails in the Motherbot’s cargo hold. Currently the Motherbot is a functional prototype. An aluminum frame for it will be built later.

2.3 Software and Communications As the motor control unit in both robots the same 16-bit microcontroller is used (see Fig. 2, Processor 2 on both robots). It is designed to be portable into just about any differential drive driven robot; its use only requires the ability to communicate through the serial port with the right protocol and a set of H-Bridges and DC-motors. Decision making and behaviors are handled in the Linux-based computers on both types of robots (Processor 1 in Fig. 2a and Fig. 2b). Hardware drivers are abstracted to a general mold, that is, they are hidden behind general interfaces for position control, ranging etc. The robot team is networked using WLAN radios connected to the Linux-based computers operating in ad-hoc mode. The routing tables are updated periodically to ensure proper connectivity. Each robot will publish its services to the network using the GIMnet software infrastructure [14] where they can be subscribed by anyone attached to the same network. This gives the robots easy access to each other’s data. Communications hubs (“tcpHubs”) will be established on each of the robots. Services include the publication of sensor data and offering robot control interfaces.

528

M. Matusiak et al.

3 Enabling Long-Term Operation: Methods and Preliminary Results 3.1 Energy and Power There are two bottlenecks for the number of robots that can be kept operational in the system: the energy consumption of Marsubots and the ability of the Motherbot to distribute energy. The battery capacities determine the recharging frequency and naturally the total operating time of the system. Marsubot: Tests were run on a Marsubot to determine the total power consumption of the robot and estimated maximum operating time. Three distinct states for robot operation were used: driving constantly with all devices on; stationary and motors turned off with all devices on; and standing by (waiting) with only the microcontroller on. The last state is used as a sleep state while its batteries are being recharged. Results are shown in Table 1 below. Table 1 Marsubot power consumption and operating time with 12 V voltage, 0.25 Ω internal battery resistance and total energy of 97.2 Wh Robot State

Current

Total Consumption

Operating Time

Driving Stationary Standby

1.3 A 0.6 A 0.1 A

16.4 W 7.3 W 1.2 W

5.9 h 13.3 h 81.0 h

Operating times are theoretical maximums not taking into account the change in battery life over many battery charge and discharge cycles. By recharging all the Marsubots once or twice, overnight operation system seems feasible. However, during most missions the robots will not be moving around all the time, so an estimate for the normal power consumption might be between the consumptions of the stationary and driving states. The second limiting factor in the operation of the society is the ability of the Motherbot to distribute energy amongst the Marsubots – currently a pulsed constant current recharge of 4 A is used with recharge and rest cycles of 5 s and 1.5 s respectively, which results in a mean current of approximately 3.1 A. Recharging is handled by a microcontroller controlled current regulating recharger (Processor 3 in Fig. 2b). Three rechargers reside in the Motherbot, one for each robot to be charged. With 3.1 A and the battery capacity of 7.1 Ah a recharge time of 2.3 h for a full charge cycle is expected. Optimization of the scheduling of the recharging time and order remains future work. Preliminary calculations with the assumptions of constant recharging energy, infinite Motherbot energy, no Marsubot power loss on recharging, maximum Marsubot energy consumption (driving mode, see Table 1) and Marsubots starting out with full batteries imply that twelve Marsubots can remain operational in the

A Novel Marsupial Robot Society: Towards Long-Term Autonomy

529

system for more than 12 h with a recharge cycle of one hour for a group of three robots. However, the scenario presented above is very much the best case scenario calculated with well functioning batteries. With the authors’ long hands on experience with battery operated systems, a worst case scenario of halving the energy capacity of the Marsubots’ batteries seems feasible (increased internal battery resistances and power losses resulting from them are left unconsidered for now). By shortening the recharge time to half an hour and given otherwise the same assumptions, a system consisting of twelve Marsubots can still remain functional without robots running out of energy at any point. Since the system will consist of only ten Marsubots, the energy distribution should be efficient enough to guarantee long-term operation of the society. The Motherbot: The Motherbot currently has four battery packs of lithium ion cells connected in parallel. The nominal voltage of one pack is 28.8 V and the capacity is 6.9 Ah for a total energy capacity of around 200 Wh (800 Wh combined in all of the packs). Four more packs will be added in the near future. A rough estimate for the power consumed in the operation while constantly active is around 100 W. Including the planned additions to the battery configuration the Motherbot could remain operational for 16 hours not counting the energy transfer to Marsubots. There are also plans for adding a fuel cell to the final robot. Energy transfer among Marsubots: Energy transfer with two Marsubots is done by connecting the robots’ batteries in parallel. It will be mainly used in recovering a robot with empty batteries by giving it enough energy to reach the Motherbot for recharging. An experiment between two robots was conducted by measuring the current after connecting the batteries. The dead robot had a battery voltage of 10.70 V at the beginning, while the rescuing robot had 13.35 V. See Table 2 below for the ampere and watt hours transferred at given times. Table 2 Energy transferred during Marsubot energy exchange (assumption of 12V battery voltage) Recharging time

1500 s

2000 s

3000 s

6000 s

Ah transferred

117 mAh

134 mAh

157 mAh

186 mAh

Energy transferred

1.4 Wh

1.6 Wh

1.9 Wh

2.2 Wh

It is believed that due to the resistances between the batteries of the two robots, the amount energy transferred between the two robots was quite low. However, even with just an extra 1.6 Wh, the robot is able to be active up to 6 minutes. This might be enough to reach the Motherbot or at least communicate with it. Implementing a recharger similar to the one used in the Motherbot in Marsubots will also be considered. However, practical experiments with Marsubot recovery remain future work.

530

M. Matusiak et al.

3.2 Connectors for Energy Transfer There are three different mechanical connections for transferring energy on the Marsubots and three recharging stations in the Motherbot, shown in Fig. 4 below. The beak’s power state is controlled with a relay. The beak is voltage sensitive, enabling a recharging robot to measure the battery voltage of the other robot. The rear connector is protected by its conical shape. The bottom connectors for connecting to the recharging rail are protected with a diode, but battery voltage can be measured externally due to a resistor connected in parallel with it.

Fig. 4 Connectors for energy transfer in the multi-robot system: left: Marsubot beak; center: Marsubot rear; right Marsubot on the Motherbot’s energy rails

3.3 Docking Robust docking is critical for successful missions. If the robot fails to dock successfully it cannot recharge. Worse still, it could get stuck and prevent other robots from docking. To assist in robust docking aligning rails have been built on to the ramp (see Fig. 1 and Fig. 5). In order to dock, a Marsubot needs to first roughly locate the Motherbot. Once the Motherbot is located the Marsubot can home in on the ramp that leads inside the Motherbot. The ramp has two rails that fit between the Marsubot’s wheels and body. Thanks to the rails Marsubot only needs to hit a single point in front of the rails and then continue straight ahead irrespective of the angle of attack. See Fig. 5 for pictures from the Marsubots camera during an automated docking run.

Fig. 5 Pictures from Marsubot’s camera as it docks (from left to right): 1. Motherbot located; 2. heading towards ramp; 3-4. rails align Marsubot

A Novel Marsupial Robot Society: Towards Long-Term Autonomy

531

4 Discussion and Future Work In this paper a novel marsupial multi-robot system, the Marsubot Society, has been introduced. One of the goals in the system’s design has been to provide a flexible platform for testing various algorithms and control schemes. This was hoped to be achieved by building a system which can run for extended periods of time without the need for user interference for recharging batteries. The hardware design and novel energy exchange qualities of the system are detailed, as well as its energy characteristics. Rescuing robots that have empty batteries to enables them to return as functional members of the Marsubot Society. The energy properties of the system have been verified with preliminary calculations and stationary tests. Also a scheme for Marsubot to dock with the Motherbot is presented. Furthermore, the introduced system has ad-hoc networking capability. This allows dynamic network topologies and bridging of data through multiple hops to recipients otherwise out of a single node’s range. Future work concerning the hardware and electronics include building a final version of the Motherbot by adding energy, sensors and a new aluminum hull. Marsubots require some development like speeding up the energy transfer between two Marsubots, but mostly the work will be in assembling rest of the members of the society. On the software side the future work holds many possibilities, ranging from distributed SLAM to multi-robot path planning and task allocation. The Marsubot Society provides an opportunity to test existing and future multi-robot control algorithms, which have so far been seen mostly in simulation studies. The optimization of energy usage and distribution in the system presents interesting opportunities in optimizing the overall behavior of the system. Other optimization criteria could include radio signal strength and the importance of communications. Another topic involving communications is network preservation and the reintroduction of lost robots. Furthermore, camera based robot docking particularly in the Marsubot to Marsubot case needs work and testing. Acknowledgements. This research was supported by the Academy of Finland.

References 1. Halme, A., Jakubik, P., Schönberg, T., Vainio, M.: The concept of robot society and its utilization. In: Proc. of the IEEE/Tsukuba International Workshop on Advanced Robotics, pp. 29–35 (1993) 2. Vainio, M., Halme, A., Appelqvist, P., Kähkönen, P., Jakubik, P., Schönberg, T., Wang, Y.: An Application Concept of an Underwater Robot Society. In: Asama, H., Fukuda, T., Arai, T., Endo, I. (eds.) Distributed Autonomous Robotic Systems 2 (DARS 1996), pp. 103–114. Springer, Heidelberg (1996)

532

M. Matusiak et al.

3. Vainio, M., Appelqvist, P., Halme, A.: Control architecture for an underwater robot society. In: Parker, L.E., Bekey, G., Barhen, J. (eds.) Distributed Autonomous Robotic Systems 4 (DARS 2000), pp. 15–24. Springer, Heidelberg (2000) 4. Appelqvist, P., Seilonen, I., Vainio, M., Halme, A., Koskinen, K.: Heterogeneous Agents Cooperating: Robots, Field Devices, and Process Automation Integrated with Agent Technology. In: Asama, H., Arai, T., Fukuda, T., Hasegawa, T. (eds.) Distributed Autonomous Robotic Systems 5 (DARS 2002), pp. 350–359. Springer, Heidelberg (2002) 5. Vainio, M., Halme, A., Troshin, I., Stipa, T., Seppälä, J., Pollehne, F., Bauerfeind, E., Haardt, H., Brault, P., Seube, N., Smerdon, A., Caine, S., Swale, B., Hakala, A.: Autonomous underwater multi-probe system for coastal area/shallow water monitoring (SWARM). In: Proc. of the EurOCEAN 2004 Conference, Galway, Ireland (2004) 6. Hölldobler, B., Wilson, E.: The Ants, pp. 291–292. The Belknap Press of Harvard University Press, Cambridge (1990) 7. Anderson, M.O., McKay, M.D., Richardson, B.S.: Multirobot automated indoor floor characterization team. In: Proc. of the 1996 IEEE International Conference on Robotics and Automation, pp. 1750–1753 (1996) 8. Murphy, R.R., Ausmus, M., Bugajska, M., Ellis, T., Johnson, T., Kelley, N., Kiefer, J., Pollock, L.: Marsupial-like mobile robot societies. In: Etzioni, O., Müller, J.P., Bradshaw, J.M. (eds.) Proc. of the Third Annual Conference on Autonomous Agents, AGENTS 1999, Seattle, Washington, United States, pp. 364–365. ACM, New York (1999) 9. Murphy, R.R.: Marsupial and shape-shifting robots for urban search and rescue. Intelligent Systems and Their Applications 15(2), 14–19 (2000) 10. Murphy, R.R.: Marsupial Robots. In: Balch, T., Parker, L. (eds.) Robot Teams: From Diversity to Polymorphism, AK Peters (2001) 11. Rybski, P., Stoeter, S., Erickson, M., Gini, M., Hougen, D., Papanikolopoulos, N.: A team of robotic agents for surveillance. In: Proceedings of the fourth international conference on Autonomous agents, pp. 9–16 (2000) 12. Drenner, A., Papanikolopoulos, N.: Docking station relocation for maximizing longevity of distributed robotic teams. In: ICRA 2006 Proc. of the 2006 IEEE International Conference on, Robotics and Automation, pp. 2436–2441 (2006) 13. Ngo, D.N., Schioler, N.: An Approach to Sociable Robots through Self-distributed Energy. In: 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 2192–2199 (2006) 14. Saarinen, J., Maula, A., Nissinen, R., Kukkonen, H., Suomela, J., Halme, A.: GIMnet Infrastructure for distributed control of Generic Intelligent Machines. In: Proc. of the 13th IASTED International Conference on Robotics and Applications Telematics, 6 p. (2007)

Predicting the Movements of Robot Teams Using Generative Models Simon Butler and Yiannis Demiris

Abstract. When a robot plans its actions within an environment containing multiple robots, it is often necessary to take into account the actions and movements of the other robots to either avoid, counter, or cooperate with them, depending on the scenario. Our predictive system is based on the biologically-inspired, simulationtheoretic approach that uses internal generative models in single-robot applications. Here, we move beyond the single-robot case to illustrate how these generative models can predict the movements of the opponent’s robots, when applied to an adversarial scenario involving two robot teams. The system is able to recognise whether the robots are attacking or defending, and the formation they are moving in. It can then predict their future movements based on the recognised model. The results confirm that the speed of recognition and the accuracy of prediction depend on how well the models match the robots’ observed behaviour.

1 Introduction There are many situations where it is beneficial to model and predict the behaviour of other robots in the environment. For example: if each robot is operating independently then it is important to avoid collisions by predicting each robot’s trajectory; in an adversarial setting, for example RoboCup soccer (Kitano et al, 1998), it is crucial to be able to neutralise and counter the opponents’ actions; and in environments where the robots have a common goal, co-operation is usually necessary to help solve the task, or to avoid duplicating work. Simon Butler EEE Dept., Imperial College London, UK e-mail: [email protected] Yiannis Demiris EEE Dept., Imperial College London, UK e-mail: [email protected]

534

S. Butler and Y. Demiris

The obvious way to obtain the objectives of the other robots is to simply communicate with them and ask what they intend to do. However in many situations this is infeasible, because an opponent will be unwilling to hand over its plans and tactics, or the robots may not share a common language so they will not be able to understand the request, or even when communication fails it is important to have a fallback mechanism to continue the task. Therefore, in these cases, the intentions of the robots must be predicted based purely on observation. In this work, an observation-based predictive system will be applied to an adversarial, real-time engagement between two teams. This domain is time-sensitive—in crisis situations the decision making time is crucial; there are numerous autonomous robots (which could also represent humans, vehicles or aircraft); it has a rich structure that allows intention recognition at different levels of a command hierarchy; the environment, including team objective and allocation, is dynamic and constantly under review.

2 Background It can be seen that recognising and predicting the goals and intentions of teams of robots is essentially a problem of model matching. Each robot collects information of the state of the environment and this data can be acted on by taking either a descriptive or generative approach. The descriptive approach uses the extraction of low-level features to match against pre-existing representations. For example, in the multi-agent domain, Devaney and Ram (1998) analyse spatio-temporal traces for coordinated motion, whether moving apart or together, however this may not scale easily to more complex plans. Using a similar technique Sukthankar and Sycara (2006) create spatio-temporal models to encode group behaviour, then match traces to these models. Although, again, more complex behaviours may not be able to be described using spatio-temporal models alone. In the domain of a soccer match Beetz and Kirchlechner (2005) try to extract explicit rules from training data. However, their approach could be prone to over-fitting if only a limited amount of training data is available, hence making it difficult to apply the results to classify new situations. Within the generative approach a set of latent (hidden) variables are introduced that encode the causes that can produce the observed data. Using these variables for a recognition and prediction task involves modifying the parameters of the generating process until the generated data can be favourably compared against the observed data. This approach has been used in the single-agent domain by using graphical models, such as Hidden Markov Models (HMMs) (Bui et al, 2002), or by using internal models to perform prediction through simulation (Demiris and Khadhouri, 2006; Demiris, 2007). The simulation-based approach is linked to the “simulation theory” perspective on human cognition that states that an observer’s cognitive and motor structures have a duel role: both acting overtly, and simulating and imagining actions and their

Predicting the Movements of Robot Teams Using Generative Models

535

consequences (Hesslow, 2002). A number of architectures have been inspired by this. One such approach (dubbed HAMMER) uses a hierarchical system of inverse (plan) and forward (predictor) models, with each model pair (collectively known as internal models) being configured to a particular action (Demiris and Khadhouri, 2006). The system finds the matching action by using the model that has the lowest error between the predicted state from each forward model and the observed state. This architecture has the advantage that it works well in on-line situations because it will always produce the most closely matching model at any point, rather than other approaches which make a selection then backtrack if it fails. A review by Demiris (2007) shows the applicability of the use of these internal models to multi-agent systems. Related work by Tak´acs et al (2007) on segmenting spatio-temporal traces of multiple agents paves the way for applying inverse models to distinct groups of agents.

3 Objectives The goal of this work is to investigate the simulationist generative approach in multiagent systems: whether an opponent’s intentions can be inferred by creating plausible plans (or hypotheses). These plans are, as time progresses, simulated to find the closest match with the observed behaviour. Subsequently, the objective that was used to create the best-matching plan is used as an estimate of the intent of the opponent. This can be thought of as prediction through the generation of multiple competing plans. In the chosen scenario for this work, two opposing teams of heterogeneous robots are fully observable within a simulated, large-scale, outdoor environment, and each opponent may have many possible objectives at different levels of abstraction. Such objectives vary from “eliminate the opponents’ bases” to “defend areas of strategic importance” at the high level, and at the low level the objectives are of the form “attack target with a wedge formation” or “surround target”. It is assumed that models exist for generating plans for the commander’s own team, and therefore hypotheses for the opponent’s team are created by applying these models to the opponent’s robots. In other words, taking a “what would I do in that situation” approach (Demiris, 2007; Demiris and Khadhouri, 2006). Additionally, the use of generative models has the advantage that they can be used not only to recognise and predict, but also to produce movement if the models are applied directly to a robot’s motor system.

4 System Architecture Starting at the lowest level, taking a generative approach to recognition and prediction of plans requires a method to generate and evaluate certain primitive actions that a team of robots can perform. The biologically-inspired HAMMER architecture provides a starting point for the system (see figure 1). It is comprised of three main

536

S. Butler and Y. Demiris

Fig. 1 The HAMMER architecture. Multiple inverse models receive the world state and suggest possible commands (C1 -Cn ), which are formed into predictions of the next world state by the corresponding forward model (P1 -Pn ). These predictions are verified on the next time step, resulting in a set of confidence values

components: the inverse models (plan generators), the forward models (predictors) and the evaluator (Demiris and Khadhouri, 2006; Demiris, 2007). An inverse model takes the current world state and an assigned goal, and outputs the required waypoints and robot parameters that, under some specified constraints, it believes are necessary for each robot to achieve the goal. Each parallel instance of a plan is paired with an instance of the forward model that provides an estimate of the events that will occur if the generated plan is followed. At each time step this estimate is returned to the inverse model to tune any parameters of the actions to achieve the desired goal. To determine which of these inverse/forward-model pairs most accurately describes the events that are occurring, periodically the output of each forward model is compared with the actual world state. These comparisons result in confidence values that behave as an indicator of how closely the observed events match each particular prediction, and they are subsequently accumulated over time until such a point that one model pair achieves a clear separation from the others. This model can then be simulated further into the future to provide a prediction of upcoming events.

5 Implementation The implementation of the system is divided into three main sections (as shown in figure 2): the simulators that host the human-controlled teams of robots; the simulators running in a slave mode, executing the internal models; and the evaluator that bridges the two, sending the state of the human-controlled world to the slave simulators and subsequently gathering and analysing their predictions.

5.1 Simulator When robots interact with each other, they can affect the state of the environment and of themselves, for example a tackle in RoboCup or the destruction of a vehicle in a military scenario. Hence, for the implementation of our architecture, we obtain the “real-world” state from humans commanding teams of robots within a

Predicting the Movements of Robot Teams Using Generative Models

Fig. 2 System implementation. This shows the human-controlled simulator instances at the top of the diagram, feeding the system state into the evaluator, which sends it to each inverse/forward-model pair and receives the corresponding predicted state. These states are then compared to get a confidence for each inverse model

537

Fig. 3 Simulator screenshot. This shows the interface for the human-controlled teams. The status of each robot is shown down the left, and there is a minimap showing the locations of the robots in the bottom right

realistic, distributed, computer-simulated environment. The engine of the simulator is based on Delta3D (Darken et al, 2005), which is an open-source project to integrate various software libraries, such as Open Scene Graph (OSG), Open Dynamics Engine (ODE), Character Animation Library 3D (Cal3D), Game Networking Engine (GNE), etc., into a coherent platform for simulation and games. The 3D engine (OSG) was used to model a large outdoor terrain (see the screenshot in figure 3), with the height and other features (texture, trees, buildings, etc) of the terrain being displayed based on 2D feature maps. The physics engine (ODE) was used to accurately model the movement of the various agents and vehicles, with additional control of their aiming and firing mechanisms. The noise introduced by the physics engine provides a stochastic element to the outcome of scenarios. When using the simulator, the human commanders are responsible for choosing goal positions for their robots, so they are free to perform manoeuvres and formations as they see fit.

5.2 Distributed Computing Infrastructure A peer-to-peer approach was taken to allow each instance to simulate the local commander’s robots (as can be seen at the top of figure 2). Positional events, and other events such as projectile detonations, are sent on each frame of the simulation over the network to the opponent, and representations of those events are shown on the opponent’s instance. For example, a 3D mesh of an opponent is moved to the position that was computed on the opponent’s instance.

538

S. Butler and Y. Demiris

Additionally, a distributed, network-based, approach was used to implement the predictive component of the system, which alleviates some of the computational burden of the execution of many models in parallel. This means that each instance of the simulator can be run on a different PC, or many simulators run on a multi-cored PC. A separate program on the network acts as an evaluator, sitting between the “real world” simulation and the slave client simulators that are acting as inverse/forwardmodel pairs. Detailed state information is obtained from the world simulation periodically. This state is used to initialise the slave clients and commands are sent to activate the desired inverse model. The simulation is run at a faster than real-time and the predictions fed back to the evaluator. Due to the faster simulation, the result of many different commands can be predicted before needing to be compared with the actual state.

5.3 Evaluation Process To analyse each inverse/forward-model pair, the evaluator calculates the normalised vectors of the movement from the robot’s previous position to both the predicted the actual positions. The dot product of these vectors is then taken and scaled by the shortest vector, as a proportion of the longest vector to give the confidence of that prediction. This metric has the desired characteristics that if the robot moves towards or away from the predicted position then the confidence approaches 1 or -1 respectively, or if it moves perpendicular to the predicted position then the confidence is zero. An additional condition was added so that if the magnitudes of both vectors are less than one metre then the confidence is 1, regardless of heading. This is to reduce errors from the case where the robot drifts or slides slightly and therefore requires a dead-zone with a radius of 1m, within which the robot is counted as stationary. Therefore the confidence, c, can be expressed as  1 if | a| < 1 and | p| < 1, c= min (| a| ,| p| ) aˆ · pˆ max otherwise. (| a| ,| p| ) where a is the vector from the start position to the actual position and p is the vector from the start position to the predicted position, and aˆ and pˆ are the normalised vectors.

6 Experiments and Results The experimental setup consists of two human commanders each running an instance of the simulator and controlling their (red or blue) team. The state of each team is sent over the network to each instance, and also to the evaluator. In this case the predictive system was applied against the blue team’s robots (the opponent). To test the feasibility of the generative approach, several inverse models were created. Each robot can have both a manoeuvre model and a formation model

Predicting the Movements of Robot Teams Using Generative Models

539

applied to it. The manoeuvre inverse models that are available for execution are either attack or defend, these are defined as follows: Attack. The robot finds the nearest enemy robot and moves towards it until it is in “weapons” range and has clear line-of-sight. At this point it stops and “fires at the enemy”. Defend. The robot finds the nearest enemy robot, and if it has clear line-of-sight then the robot moves in the opposite direction until hidden from its sight. The formation inverse models are either wedge, circle, line or column and are shown in figure 4. Each formation is described by a weighted graph (as defined by Ji and Egerstedt (2007)), so each robot must keep a distance (scaled by the weight) from their neighbours and this distance is assumed to be a constant 50m for this experiment. Therefore their target position is a linear combination of the offsets to each of the positions required to maintain the distance to each of the connected neighbours. Each formation requires a robot to act as the leader and this was implemented by using a heuristic measure to select the robot that is nearest to the target position. This then allows the other positions to be assigned by nearest neighbour, starting with the leader.

Fig. 4 Weighted graphs describing each formation. From left to right: wedge, circle, line and column. The numbers indicate the weight of the connection, higher numbers mean increasing repulsive forces. Those with just one weight specified indicates that all the weights take this value. The darker grey circle indicates the robot that acts as the leader of the formation

The evaluator launches two instances of the simulator running in a slave mode, and sets a queue of models to simulate against the opponent’s blue robots. Due to our approach, the manoeuvre and formation models cannot be treated independently, therefore all combinations must be evaluated. This is because the specific movements of the robots in formation depend on the the route determined by the manoeuvre model. For example, features of the terrain affect the speed of each robot, and any local object avoidance will cause the formation to be disrupted. However, these disruptions can be accounted for by simulating the formation along the route determined by the manoeuvre model. Therefore on one instance the attack model with each of the formations is simulated and on the other the defend model is used, again with each of the formations. For ease of analysis, there are three robots on the blue team and one robot on the red team. Each slave simulator is initialised with the world state then run for a prespecified duration (five seconds) at an increased time scale (4x) and the positions of the robots are returned to the controller within two seconds. The instance is then

540

S. Butler and Y. Demiris Trace of the robots’ x−y movements on a contour map of the terrain

400

Accumulated average confidences of different models

START

300

Attack:Wedge 7

90

0 600

200

8

00

80

29

Attack:Circle Attack:Line

96

Attack:Column

6

6000

101

Defend:Line

0 600

4000

−100

0 00

4

−200

103 END/DTD END/DTD 88 99 END/DTD 100

−300 −400

−600

74 84

71

59

−50

44

54

30

40

0

START

3

1

55 40 START

−100

Defend:Column

4

2

70

START

50 Metres

2000

4000

−500

85

Defend:Circle

5

6000

END/DTD

0 Metres

Defend:Wedge

106

Confidence

100

0

100

−1

150

Fig. 5 Unit movements. A contour map (dashed lines), and a trace (thick lines) of the movements of blue robots (starting at the bottom, in a valley), and the red robot (starting at the top, on a ridge). The numbers on the traces represents the time step at that point, and the numbers on the contours represent the height of the terrain

0

20

40

60 Time step

80

100

120

Fig. 6 Accumulated confidences of each model Showing the attack and defend inverse models, and the formations wedge, circle, line and column applied to each. The change in formation at timestep 60 is not clear

Accumulated confidences of manoeuvre models 10 Attack Defend 8

Confidence

6

4

2

0

−2

0

20

40

60 Time step

80

100

120

Fig. 7 Accumulated confidences for attack or defend models. Using the average confidence of the leader, a clear separation can be seen between attack and defend

Fig. 8 Accumulated confidences for formation models. Using the average confidence of the formation excluding the leader, the change from wedge to line formation can be seen at points A and B respectively

reset and the process repeated with the next model in the queue, upon the receipt of the next world state. Therefore a result for each model is obtained every 8 seconds. As can be seen from the trace of each robot’s positions for this experiment (shown in figure 5), the red team’s objective is to at first to stay hidden behind a ridge, then once the opponent’s robots are close enough, move over the ridge and attack. The blue team’s robots move together in an attacking wedge formation, towards the hidden red robot. When they move closer to the red robot, they move into a line formation (at around timestep 60). Then when the red robot comes into range they stop and fire at it, until it is destroyed.

Predicting the Movements of Robot Teams Using Generative Models

541

The overall confidence of each combination of inverse models is calculated by averaging the confidence of each individual robot within the team, repeated for each simulator instance. The accumulated confidence values are shown in figure 6. From this we can see the separation between the attacking or defending models, but the change in formation that occurs at around timestep 60 is not clear. This is because the averaging of the confidence takes place over all the robots in the formation, however, the leader of the formation will always have the same confidence regardless of the formation being performed. Therefore we can separate the inverse models by averaging the confidence values of the leader in all the formations for each manoeuvre model (shown in figure 7) and by averaging the confidences of the remaining robots each of the formations (i.e., excluding the leader) that apply to the winning manoeuvre model (shown in figure 8). Now the change in formation that occurs around timestep 60 can clearly be seen by the slowing of the confidence of the wedge formation, and the gaining in confidence of the line formation.

7 Conclusions and Future Work The work reported here constitutes a first study into the “simulation theory” approach to intention recognition when applied to the multirobot domain. Our results indicate that such approach holds good promise as it is able to recognise and predict whether robots are attacking or hiding from each other, and the formation that they are following in a simulated adversarial scenario. The system is also able to predict the future positions of the robots, assuming the same models hold. It is still an open problem of how to best combine the confidences of each individual robot into an overall confidence for the inverse model. Immediate extensions include the parameterisation of the inverse models so that if, for example, a robot is moving slowly due to damage, the speed of the robot set by the inverse model can be tuned to match. This tuning can be based on the feedback of the confidence of the previous prediction to the inverse model. A drawback of this approach is the high computational cost required to evaluate the inverse/forward-model pairs for each possible hypothesis. There are several solutions to this, such as, assigning preconditions to each of the models so they are only executed under certain circumstances, or to briefly run each model and to rank them based on their confidence to find the most promising ones. Longer-term predictions could be produced by inverse models that do not depend directly on the world state, but combine the confidence of the low-level inverse models (e.g. formations or manoeuvres) with higher-level statistics (e.g. robot losses or speed of movement), to generate predictions that have a lower spatial and temporal resolution. For example, a high-level inverse model such as “defend base” would predict that the movement of robots is low, and that they are highly concentrated, with the centre of mass being in the base, and would be reinforced if the defend inverse model has a high confidence. This can lead to a system that is useful when using the simulator to alert the commander when an attack is imminent, or to

542

S. Butler and Y. Demiris

infer the high-level intentions of the opponent, so the commander can better form an effective strategy. We are currently persuing an extension to this system that assumes that the environment is only partially observable, for example through the implementation of the architecture on our team of outdoor P3-ATs operating as a multi-robot system. This makes the deployment of sensors to detect the position of the opponent an important part of the strategy. This can be done by making assumptions on the locations of the opponent and running the inverse/forward models, then ranking the results by the threat posed by each prediction, then deploying the sensors to verify the assumptions, covering the most risky models. Acknowledgements. We would like to thank B´alint Tak´acs, Tom Carlson, Murilo Fernandes Martins and Paschalis Veskos for their comments.

References Beetz, M., Kirchlechner, B.: Computerized real-time analysis of football games. IEEE pervasive computing 4(3) (2005) Bui, H., Venkatesh, S., West, G.: Policy recognition in the abstract hidden markov models. Journal of Artificial Intelligence Research 17, 451–499 (2002) Darken, R., Mcdowell, P., Johnson, E.: The Delta3D open source game engine. IEEE computer graphics and applications 25(3) (2005) Demiris, Y.: Prediction of intent in robotics and multi-agent systems. Cognitive Processing 8(3), 151–158 (2007) Demiris, Y., Khadhouri, B.: Hierarchical attentive multiple models for execution and recognition of actions. Robotics and autonomous systems 54(5) (2006) Devaney, M., Ram, A.: Needles in a haystack: Plan recognition in large spatial domains involving multiple agents. In: National Conference on Artificial Intelligence (1998) Hesslow, G.: Conscious thought as simulation of behaviour and perception. Trends in Cognitive Sciences 6(6), 242–247 (2002) Ji, E.M.: Distributed coordination control of multiagent systems while preserving connectedness. IEEE Transactions on Robotics 23(4), 693–703 (2007) Kitano, H., Asada, M., Kuniyoshi, Y., Noda, I., Osawai, E., Matsubara, H.: Robocup: A challenge problem for AI and robotics. In: Kitano, H. (ed.) RoboCup 1997. LNCS, vol. 1395, pp. 1–19. Springer, Heidelberg (1998) Sukthankar, G., Sycara, K.: Robust recognition of physical team behaviors using spatio-temporal models. In: AAMAS 2006: Proceedings of the fifth international joint conference on Autonomous agents and multiagent systems, pp. 638–645. ACM, New York (2006) Tak´acs, B., Butler, S., Demiris, Y.: Multi-agent behaviour segmentation via spectral clustering. In: Proceedings of the AAAI-2007, PAIR Workshop, pp. 74–81. AAAI Press, Menlo Park (2007)

Interactive Mobile Robotic Drinking Glasses Franc¸ois Rey, Michele Leidi, and Francesco Mondada

Abstract. The central idea behind this attempt is to merge common objects and robotics to obtain a new type of interactive artefact, we call robjects. Robjects provide services to everyday life and can be controlled by the user in a very intuitive way. Robjects take rarely the initiative, have few decisional autonomy. Most of their activity is centered and controlled by the interaction with the user. To test this concept in a concrete experiment we decided to include mobile robotic technology into drinking glasses to improve their service on a table. Making them mobile and interactive brings a new dimension to the table, simplifies service and improves security. The system works in the following manner : when a user starts filling a glass, the other empty glasses come closer to get filled too and then go back to their initial position. Owners of the glasses can refuse to fill their glasses by simply bringing back the glass on its initial position as soon as the glass starts to move. The system has been prototyped and tested. The analysis of the interaction of some users shows strengths in interactivity and weaknesses in the speed of the system and in reliability of the actual prototype.

1 Introduction 1.1 Goal of the Project The goal of this project is to develop a swarm of “robot objects“, we call robjects. Robjects are fully autonomous robots with the shape of common objects. They interact with users in a very intuitive way to provide services in everyday life situations. Interaction is fully based on a multimodal natural interface applied to the object itself. Touching, pushing, grasping, making gestures or a sequence of actions are the Franc¸ois Rey, Michele Leidi, and Francesco Mondada ´ Laboratoire de Syst`emes Robotiques - Ecole Polytechnique F´ed´erale de Lausanne e-mail: [email protected]

544

F. Rey et al.

Fig. 1 Interactive and mobile drinking glasses merging the role of glasses and mobile robotic technology. Lateral and top views

basic elements of a simple interaction like the one we present here. The goal of the interaction is to control the robot and bring him to do what the user desires. This concept is in opposition to most domestic autonomous robot concepts (often with an humanoid form) proposed by the scientific community and fits better with people expectation, at least in Europe [1]. The scenario presented in this paper is the lunch of a family with two adults and two children. Everybody drink water and parents drink wine too. Water and wine are in glasses of different shapes. All glasses together form a swarm where each glass can move and interact with users and other glasses. When a glass is filled, it communicate this event to the swarm of glasses. The location of each glass is given by a vision system placed above the table. Glasses that have the same content and are nearly empty start moving toward the glass who communicated the event. If the glass is taken back or manipulated by an user, the action is aborted. If not, the glass continues its displacement avoiding obstacles. When sufficiently close to the original glass, they stop, show by a LED that they are ready to be filled and wait until they are refilled. Then they move back to their original position. If the glass is placed too close to the table border, the glass takes the initiative to go back to its original position. An alternative scenario involves a carafe robject. In this case the detection of the filling action is done when the carafe is taken and not when the glass starts to be filled.

1.2 Current State of the Art Very few research groups have started working on distributed robotic systems acting in the living environment, what they call ubiquitous robotics. The most active

Interactive Mobile Robotic Drinking Glasses

545

group is leaded by Prof. Kim and started to work on this topic in 2004 [2, 3, 4, 5]. Their vision of ubiquitous robotics is based on the communication between intelligent software agents (Sobots), sensors networks (Embots) and classical mobile robot (Mobots). Sobots represent the intelligence, Embots the sensor part of the system and Mobots the actuator part. User interaction is not playing a central role in their concept. The combination of these domains is apparently only a way of generating a new application field for the three technologies, without a real integration. Another attempt at defining ubiquitous robotics has been made by Takeda [6] with a stronger emphasis on interaction. This paper introduces a very interesting vision including specific interaction modes such as ”intimate interaction“, ”loose interaction“ and ”cooperative interaction“. This work has been limited to some experiments, without a real integration effort and a development of concepts and technology. Nobody has developed mobile drinking glasses, but some systems look for similar services, like the BrewskiBot of MobileRobots Inc. It’s a mobile fridge which brings some drink to people. In fact it’s a little fridge fixed on a PatrolBot, a robot design to patrol in an area.

2 The Mechatronic System 2.1 Specifications To carry out the selected scenario, our robjects must respect the following specifications. • • • • • • • • • •

To contain 3 dl of liquid. Its form has to be adapted to the content. To move without spill the content. To draw the attention with a luminous or sonorous signal. To detect an impact with an acceleration higher of 1g (10 m/s2 ) and its direction with a precision of more or less 10◦ . To detect an obstacle placed within 2 centimeter around the robot. To orient itself according to the carafe and the user with a precision of 2 cm and about 5◦ . To detect liquid level in the glass (about ten different levels for a glass of 30cl). To detect when someone lift up the carafe. To detect the border of the table in order to avoid falling from the table. To detect when the glass is too close (less than 15 cm) to the table border.

2.2 Selected Solution After an analysis of several options and constrains of the project (duration, competences involved) we decided to stick a common drinking glass on the top of a flat mobile robot. The robot has two differential wheels to move on the table. We use LEDs to draw the attention of users. The robot has an accelerometer sensor to detect

546

F. Rey et al.

Fig. 2 Structure of the system with a PC controlling the robjects and observing their position using a camera

PC

video orders via bluetooth feedback via bluetooth

impacts. Around his body, height infrared proximity sensor are available to detect obstacles. We use a tracking system based on the ArTag Library [7] and a high definition camera connected with a computer to detect the positions of the robots. The computer can exchange information with the robots using Bluetooth devices. We use a force sensor as a balance to check the liquid level. The carafe is also placed on a robotic element to detect when someone lift it up. This robotic element of the carafe has a force sensor and can exchange information with Bluetooth but cannot move. The table border is detected by the high-definition camera which is already used to find the position of glasses.

2.3 Structure of the System Despite the long-term purpose to design a fully distributed system, this first attempt has been based on a centralized approach, at least at the structural level. A PC has the control of the whole setup, taking images from the camera, extracting the position of the glasses in respect to the table shape and controlling the glasses based on the feedback they give (see figure 2). The glasses robjects are still in charge of all sensor pre-processing, detection of events and motor control.

2.4 Mechanics of the Robjects The robject has the following configuration: It is a glass fixed on a small cylindrical mobile robot having 10cm of diameter and 3.4cm high (figure 3). The robot has two differential wheels with a diameter of 2.6cm. The part of the robot that is glued to the glass can move around an axis and acts on a force sensor as a balance. The same mechanical part protects the electronic of spatter. Figure 4 shows the mechanism of the balance. A plexiglas frame (blue part in figure 3) fixes the battery, motors and the print circuit board. Wheel are actuated by stepper motors with 20 steps by revolution and a 50:1 gearbox. The torque is sufficient to move the robot with a payload of one kilogram.

Interactive Mobile Robotic Drinking Glasses

547

Fig. 3 Proe drawings of the mecanical part Fig. 4 Geometrical configuration of the balance

2.5 Electronics of the Robjects The electronics of the robot is based on the electronics of the e-puck robot1 . The microcontroller used for controlling the robot is a DSPIC30F by Microchip running at 15 MIPS. It has 68 I/O pins of which 12 can perform analogical/digital conversion and it support two UART communications channels. To detect obstacles, we use infrared proximity sensors. Height proximity sensors are placed symmetrically to the forward axis with angles of 20◦ , 45◦, 135◦ and 160◦. It could have been useful to place a sensor at 90◦ but unfortunatly wheels prevented from placing the sensor at this position. The frame’s plexiglas sidetracks a part of the infrared sensor’s ray which is reflected on the floor. This can cause troubles if the distance between the plexiglas and the floor changes, because the sensor’s value changes too. We exploited this aspect to detect when the plexiglas has passed the table’s border. To measure the liquid level and to know when the glass is lift up we use the “millinewton” force sensor [8]. This sensor returns a voltage functions of the force applied on a contact ball. The sensor saturates with a force of about 2N. To know the position in space of the robot and to detect impacts we use a 3D accelerometer sensor. With this sensor and the force sensor we can know if the glass is lifted by the user. To communicate with the user, the robot has eight horizontal red LEDs on the top and four vertical yellow LEDs. We can command each red LEDs individually. All yellow LEDs are controlled together. To exchange information with the computer or receive order from the computer, the robot has a Bluetooth device. The robot has a lithium-ion battery for supplying with a voltage of 3.7V and a capacity of 1400 mAh. With this battery the robot has an autonomy of about two hours when the motors are continously active. 1

http://www.e-puck.org

548

F. Rey et al.

Fig. 5 CAD drawings of the pcb’s top and pcb’s bottom

3 Software The main software runs on the PC computer and receives information from the robjects and from the high-definition camera fixed over the table. From camera images we extract the location of all glasses and of the table border. Using bluetooth, the PC sends orders to the robots as function of the received information obtained by the camera and from the robots. The software running on the robots receives and executes order received from the PC. Some basic algorithms to do obstacle avoidance, table’s border detection, pre-processing of force sensor data (if someone takes the glass, if the glass is empty, if someone fill up or drain the glass, if the liquid level laze) are embedded in the robots themselves.

3.1 Image Processing and Selected Camera To know the location and the direction of each glass and of the carafe, we apply visual tracking using the ArTag library [7]. This library was mainly developed for augmented virtual reality and works with 2D tags which are found in the image and identified. We can create objects with many tags having a fixed geometrical relation among them. The software must recognize at least two tags to identify an object and its location. The robots have twelve tags on the tray around the glass; with this configuration we expect that the camera can see at least two tags which are not hidden by the glass itself or by the user. Unfortunately at this location there is not a lot of space and tags are small (15 x 15 mm). This size defines the required resolution of the camera: if we want to analyze a square table with a size of 1.5 x 1.5 m, we must use a camera with at least a resolution of 2000x2000 pixels. To minimize radial distortion we use a lens with a focal of 12mm and we choosed an integration time of 250 milliseconds to increase picture quality. With this configuration, the delay between image acquisition and position computation is about one second. Therefore odometry is combined with the tracking system to get real-time position values.

Interactive Mobile Robotic Drinking Glasses

549

3.2 Embedded System The software running on the robots allows them to autonomously (i) move forward for a given distance avoiding obstacles and table’s borders, (ii) turn for a given angle, (iii) switch on or off LEDs, (iv) calibrate proximity sensors, (v) calibrate the balance and sent the balance’s state (empty, carried, filling up, emptying, stagnant) or the balance’s value, and (vi) send the motion detector’s value.

3.3 Main Control The main control runs on the computer, manages all the system and send orders to the robots by Bluetooth. When launched, the software starts by performing an initialization of the camera and of the ArTag library, a connection with the glass robots and the carafe robots, detection of the table border and of the robots location. Then the software runs in a loop and controls each glass as following: • If the glass is at its location and is empty, the software check if someone fills up another glass (A). In this case the software sends orders to move the glass toward the glass (A). • If the glass is not moving and somebody moves it, the software controls the new location. If the glass is too close to the table border, the software sends the order to move the glass toward is last location, otherwise it record the new location. • If the glass is moving to be filled up and somebody takes it, the software checks the new position. If the glass is nearer to its standard location than to the place where it must be filled up, the software send the order to move back to its standard location. Otherwise the glass will continue its displacement. • When the glass reaches the place where it must be filled up, it waits for somebody filling it up and then returns toward is location.

Fig. 6 On the left: the glass 1 is filling up and the two empty glasses move to it, On the right: the glass 3 was filling up and returns to its location

550

F. Rey et al.

Fig. 7 Somebody moved glasses 1 and 2. Glass 1 is too close to the table border and returns to its last location, glass 2 don’t move and record its new location

4 Validation in Real Situations Three experiments were done on a square table with a size of 1.2 x 1.2 meters. The first experiment was done with three persons who had each a glass of water. The second experiment has been carried out with four persons with a glass of water each. The third experiment looked to four persons as in the original scenario, two having only a glass of water, two having both water and wine glasses. Users had no idea about how the system works and got only few explanations (one minute) before the start of the experiment.

4.1 Users’ Feeling Users have given their feeling about the system, mainly on usefulness, readiness and speed of the system. • It is easy to interact with the glass. The action to catch the glass when it moves toward the filling place if the user don’t wont to drink anymore is really intuitive. It’s nice to see the glasses coming when somebody start filling. It’s natural. • The system is not fast enough. A speed of 8 cm/s is not sufficient. The speed of mobile robots when interacting with humans is a common problem in interaction with robotics infrastructures [9]. In our case, when somebody finished to fill up his glass, the other glasses were still at the half of their trajectory. This delay destroys the efficiency of the interaction. • Wrong robot’s actions (avoiding non-existent obstacles, ...) distract the users who don’t know how to react. • Since the glass is glued on the robot, users have to lift all the system, which looks strange because of the impression that the robot could fall. Users get quickly used to this wrong impression. • This system don’t allow to follow the civility’s rule which says that we must serve other persons before serving ourself, or, like in Japan, that we cannot serve ourself!

Interactive Mobile Robotic Drinking Glasses

551

5 Conclusion The positive conclusion of our experiments is that the robject concepts seems very interesting and allowing natural interactions between robots and users in everyday life. The negative aspect is that this type of interaction must be precise, reliable and fast. This is still not the case of our system. The main interaction problem is clearly generated by wrong robots’ actions: sometimes robots misinterpret signals and react in the wrong way, generating confusion in the user. That’s due mainly to the simplicity of the embedded sensors. Another big problem is the speed of the system, not sufficient to implement an optimal interaction. On the other side, this project demonstrates that we can use simple, intuitive and usual actions to control robots. For example filling up glasses, catching a moving glass, placing it on standard positions etc. This interaction allows everybody to accept and use robotic technology embedded in everyday objects. Acknowledgements. We would like to thanks Luc Bergeron and Fr´ed´eric Kaplan for their suggestions, ideas and support, Tarek Baaboura for parts manufacturing, ACI staff particulary Peter Br¨uhlmeier for pcb routing and Andr´e Badertscher for components soldering, Michael Bonani for advice in electronic, and all testers for their patience and very valuable feedback.

References 1. Ray, C., Mondada, F., Siegwart, R.: What do people expect from robots? In: IEEE/RSJ 2008 International Conference on Intelligent RObots and Systems (2008) 2. Kim, J.-H.: Ubiquitous robot. In: Computational Intelligence, Theory and Applications, Keynote speech paper of the 8th Fuzzy Days International Conference, pp. 451–459. Springer, Heidelberg (2004) 3. Kim, J.-H., Kim, Y.-D., Lee, K.-H.: The 3rd generation of robotics: Ubiquitous robot. In: Proc. of the International Conference on Autonomous Robots and Agents (Keynote Speech Paper) (2004) 4. Kim, J.-H.: Ubiquitous robot: Recent progress and development. In: ICASE International Joint Conference 2006 (Keynote Speech Paper), pp. I–25 – I–30 (2006) 5. Kim, J.-H., Lee, K.-H., Kim, Y.-D., Kuppuswamy, N.S., Jo, J.: Ubiquitous robot: A new paradigm for integrated services. In: 2007 IEEE International Conference on Robotics and Automation, pp. 2853–2858 (2007) 6. Takeda, H., Kobayashi, N., Matsubara, Y., Nishida, T.: Towards ubiquitous human-robot interaction. In: Working Notes for IJCAI 1997 Workshop on Intelligent Multimodal Systems, pp. 1–8 (1997) 7. Fiala, M.: Artag, a fiducial marker system using digital techniques. In: CVPR (2), pp. 590–596. IEEE Computer Society, Los Alamitos (2005) 8. Birol, H., Maeder, T., Nadzeyka, I., Boers, M., Ryser, P.: Fabrication of a Millinewton Force Sensor Using Low Temperature Co-fired Ceramic (LTCC) Technology. Sensors and Actuators A: Physical 134, 334–338 (2007) 9. Mondada, F., Legon, S.: Interactions between art and mobile robotic system engineering. In: Gomi, T. (ed.) ER-EvoRob 2001. LNCS, vol. 2217, pp. 121–138. Springer, Heidelberg (2001)

Part VIII

Human-Robot Interaction

Adaptive Supervisory Control of a Communication Robot That Approaches Visitors Masahiro Shiomi, Takayuki Kanda, Kenta Nohara, Hiroshi Ishiguro, and Norihiro Hagita

Abstract. In the future, we believe communication robots will be working in shopping malls, talking to people, and providing information for them. In a mall, a robot must choose appropriate visitors as targets to approach. But since robotics technology has not matured yet, fully autonomous communication robots that can deal effectively with problematic situations are not expected; annoying robots who obstruct people in malls must be avoided. Instead, we chose a supervisory control scheme where a robot requests permission from a human operator to approach selected visitors. To reduce the time required by the operator to give approval, the system adaptively learns the operator’s choice to reduce the number of approval requests based on the classification of visitor walking behaviors and their attitudes toward the robot. We evaluate the effectiveness of our adaptive supervisory control system in a field trial at a shopping mall. The experimental results demonstrated that 86.5% of the targets requested by the system for approval were appropriate. Moreover, operating time was reduced: the operator only controlled the robot 34.9% of the experiment time.

1 Introduction Our research has developed ”communication robots” that are capable of natural human-robot interaction (HRI) and that can support human activities in daily environments [1, 2]. For instance, a humanoid robot in a mall might provide information about shops and restaurants by effectively using both verbal and non-verbal Masahiro Shiomi, Takayuki Kanda, and Norihiro Hagita ATR-IRC e-mail: {m-shiomi,kanda,hagita}@atr.jp Kenta Nohara and Hiroshi Ishiguro ATR-IRC and Osaka Univ. e-mail: nohara,[email protected]

556

M. Shiomi et al.

expressions [3, 4]. However, as inspiring as seeing such robots operating in a mall might be, the inescapable reality is that social dynamics are complex; current technology cannot yet support a fully autonomous robot that plays a real role in society. Any robot will eventually find itself in unforeseen circumstances, where failure to respond appropriately could lead to social awkwardness, financial loss, or even dangerous situations. Instead, our solution uses semi-autonomous communication robots that are partly supported by human operators [5]. Behind such immature technology, this approach commonly places a human operator who helps the system when it can not work autonomously. This technique has been used in tele-robotics and networked robotics, mainly focusing on robots for space and other explorative tasks. In human-robot interaction, recent field trials in social settings have involved some degree of remote control [6, 7] or close surveillance. The semi-autonomous approach allows communication robots to capably interact with people to achieve useful tasks in real fields. Moreover, it enables human developers to continue to improve robot capability so that the robots can gradually become more autonomous [8]. This paper focuses on the ”approaching” situation. In a public setting, the research question concentrates on whom to approach and whom to avoid. In Figure 1, a robot approached a visitor to invite him to the shop. Appropriate decisions are critical so that the robot is useful. Of course, approaching all visitors is inappropriate and impossible. In a mall, people who are window-shopping might welcome such additional information provided by the robot, but not others who are busy or just passing by [9]. Instead of making this decision fully autonomously, our human operator chooses a target person to approach. In other words, the robot was under supervisory control. In addition, we focused on the adaptation of supervisory control. The operator continuously decides whether to approach. Here, we have a chance to increase the robot’s autonomy. Numerous studies in robotics have been concerned with ”learning by demonstration.” Our study, which concentrates on people walking in a shopping mall because their behavior has rarely been studied, establishes models of human activity by carefully studying such behavior [10]; this paper tries to learn such decision criteria about walkers in malls from a human operator.

2 Supervisory Control during Approaching At the beginning of an interaction, a robot needs to approach an appropriate person because it does not want to miss opportunities to provide service. In addition to identifying appropriate targets, the timing of the approach is important. People might ”escape” if the robot approaches too late. Satisfying these demands for completely autonomous robots is difficult, because the robot needs decision criteria to select targets for approaching in various situations. Therefore, we placed a human operator in a supervisory position over the robot to enable us to apply it in real environments without worrying about customer complaints.

Adaptive Supervisory Control of a Communication Robot

557

Fig. 1 Robot approaching and inviting a person to a shop Fig. 2 System overview

Figure 2 shows a model of our adaptive supervisory system that has two parts: a perception system (left) and an action system (right). These parts are connected by the operator. The system autonomously selects a target to approach and requests operator approval. In addition, it learns the operator’s decision to improve its choice of target. The system consists of the following components: position estimation, primitive estimation, visualization support interface, adaptive supervising-request system, and a humanoid robot. In the system, people’s behaviors are abstracted into ”primitives,” which are labeled information about positions and trajectories. For example, we often observed that a person who ”is walking slowly” and ”approaching the robot” seems to be an appropriate target for whom the robot can furnish information-providing service. Similarly, the robot could provide route-guidance for a person ”stopping” ”in front of a map.” Such walking behavior, behavior toward the robot, and location are all effective information for choosing a target. We defined people’s behavior and the location as ”primitives” that are displayed to the operator for visualization support and used for system adaptation. An adaptive supervising-request system is one essential component. The operator approves the target. His decision is simultaneously used for adaptation so that the system learns to avoid requesting approval for inappropriate targets. The operating time in supervisory control is decreased based on learning. Visualization support helps the operator decide and reduces the time required by displaying the primitives of a target request. Since the approach is very time-critical, the operator must decide quickly after the system requests supervision. As shown in Figure 3, the approaching opportunity between the robot and the target only lasts a few seconds. If the operator hesitates too long, the robot loses the opportunity to approach. Thus, decreasing the operation time for deciding is crucial. Our humanoid robot is mostly autonomous. The operator only gives permission to approach. After receiving permission, the robot approaches the target and

558

M. Shiomi et al.

Fig. 3 Illustration of timing during supervisory request

provides the pre-scripted information. When idle, the robot moves around autonomously.

3 System Configuration In this section, we describe the details of each component of the adaptive supervisory control system introduced in Section 2.

3.1 Position Estimation System We developed a system to estimate the positions of people and the robot using a Laser Range Finder (LRF) [11]. The system uses particle filters and a parametric human shape model to estimate people’s positions and estimates the robot’s position using a model made by considering its locomotion. We used six SICK LMS-200 laser scanners to scan an angular area of 180 deg. at a resolution of 0.5 deg., covering a radial distance of 8 m with a nominal system error of ± 20 mm. They provided readings of 361 data points every 26 milliseconds. LRFs were mounted at a uniform height of 90 cm, which was slightly above waist level for most subjects.

3.2 Primitive Estimation System In this study, we focused on the primitives not only in people’s behaviors but also in spatial information because the meaning of people’s behaviors changes in such specific areas as in front of a map, near a bench, and so on. From this point of view, we defined two categories of primitives: behavioral and spatial. 3.2.1

Behavioral Primitives

Behavioral primitives, which define such human states as walking, running, going straight, and turning right, consist of two kinds: walking behavior, which is independent of the robot, and behavior toward the robot, which is defined based on proximity [12] to it and the traveling direction toward it. The details of the estimating method are described in [13].

Adaptive Supervisory Control of a Communication Robot

559

We defined four categories of walking behavior and two of behavior toward the robot. In the former categories, we focused on long-term and short-term behaviors because they help the operator determine the details of people’s movements. The following are the details of each category: Style: Six classes based on the trajectory of people: straight, turn-right, turnleft, this way and that , U-turn, and stopping. A five-second trajectory is needed for estimating this category Short-style: Five classes based on a short-term trajectory: straight, turn-right, turn-left, U-turn, and stopping. A two-second trajectory is needed for estimating this category. Type: Five classes based on people’s speed: running, brisk walking, idle walking, stopping, and waiting. A seven-second trajectory is needed for estimating this category Short-type: Four classes based on people’s short-term speed: running, brisk walking, idle walking, and stopping. A two-second trajectory is needed for estimating this category. Robot-distance: Three classes based on the distance between the people and the robot: talking distance, observing distance, and far distance. A two-second trajectory is needed for estimating this category. Robot-direction: Four classes based on traveling direction toward the robot by people: approach, escaping, stopping, and unknown. A two-second trajectory is needed for estimating this category. 3.2.2

Spatial Primitive

Spatial primitives are used for defining the area’s state that depends on such elements of the environment as the front of a map on a wall or a shop, on a bench, and so on. We defined six spatial primitives for the experiment environment: the side of a wall, on a bench, a shop front, in front of the shop, the front of a map, and the robot’s moving area. The system simply classified people’s spatial primitives using x-y coordinate information.

3.3 Adaptive Supervising Request System The system learns decision criteria to request supervision from the operator to correctly select those to approach and those to avoid. For this purpose, we used a Bayesian network (BN) [14] and primitives when the operator decides to approach because BN represents the relationships between the operator decisions and primitives in a probabilistic manner using statistics. BN also enables us to easily understand the internal decision criteria. Each value of the hypothesis node comes from the primitive estimation system that calculates the value of the evidence node for people around the robot. If the calculated value is higher than a threshold, the system requests supervision from the operator through the visualization interface whose details are described in the next section.

560

M. Shiomi et al.

(a)

(b)

Fig. 4 Control software for human operator

3.4 Visualization Interface We developed an interface with which the operator provides supervisory control (Fig. 4) and approves or declines requests raised by an adaptive supervisory control system that consists of two screens: one gives a top-view image with a function to approve/decline (Fig. 4(a)), and the other gives a video image showing potential targets detected by the system (Fig. 4(b)). Visualization support is an important function of the interface for supervisory control. As discussed in Section 2, the approach is very time-critical, and the decision time, which is the time required for the operator to decide after receiving a request, is also a critical factor for system performance. If the operator’s decision is too late, the target person will already have passed the robot. Decision time must be decreased to avoid missing chances to talk to people. The interface has four functions for visualization support. The first monitors the situation around the robot and displays target positions and primitives over a map of the environment (Fig. 4(a)-A,B). The second quickly finds the person selected by the request system and emphasizes that target (Fig. 4(b)). The third informs the operator of a supervising request. When the system finds a target, it warns the operator. The fourth function passes the operator decisions to the request system; the ’yes,’ ’no,’ and ’cancel’ buttons are all connected to this function (Fig. 4(a)-C). The ’cancel’ button is used when the operator could not decide whether to approach. For example, deciding is difficult if the system selected a person standing more than 8 m from the robot. In such cases, the operator uses ’cancel’ to prevent training the adaptive requesting system.

3.5 Robovie We used Robovie, a humanoid robot, for this system. Fig. 1 shows ”RobovieIIF” [15], an interactive humanoid robot characterized by its human-like physical

Adaptive Supervisory Control of a Communication Robot

561

Fig. 5 Shopping mall map

expressions and various sensors. The robot is mostly autonomous except when starting to approach a selected target. In the idling state, the robot moves around in its environment using information from the position estimation system. If a likely target enters the environment, the system asks for permission to approach. If permission is granted, the robot autonomously approaches and gives information or guides the person with pointing gestures as well as such reference terms as ”this” and ”that” [16].

4 Experiments We conducted a field trial with our developed system at a shopping mall to evaluate our adaptive supervisory control system for communication robots during approaching situations.

4.1 Environment We evaluated our system with a field test from January 21st to 25th, 2008 at a mall located between a train station and an amusement park. Visitors were mainly families, couples, and sightseers who could freely interact with the robot. Fig. 5 shows the environment; the robot, six LRFs , and four cameras were installed at a corridor of the mall visited hourly by more than an average of 100 people. The LRF arrangement was chosen so that every point within the area of interest was covered by at least two sensors to reduce occlusions. For safety, one staff member remained around the robot during the field trial. Since at the beginning of the field trial, the BN was not trained, the operator directly selected the targets the robot should approach. Operator selections were used Table 1 Experimental results Experiment time

Time when persons are 8 m of robot

Operating time

Supervision request rate of approach targets

Error rate of supervision requests

2,414 sec.

1587 sec.

147.5 sec.

86.5%

34.6%

562

M. Shiomi et al.

for training the BN in real-time. After a few selections, the system started sending supervision requests to the operator, who stopped selecting targets and began to respond to the requests for decisions.

4.2 Results The experiments consisted of two phases: training and requesting. In the training phase, real-time training was conducted for 60 minutes. For training, the operator used 184 data sets of approach targets that consisted of ’yes’ or ’no’ labels, the time of the operator’s decision, and the behavioral and spatial primitives. The operator also gave 128 data sets of non-approaching targets. We measured the operating time in the requesting phase; the experiment was conducted for 2414 seconds (about 40 minutes). We also measured the time someone was in the environment within 8 m of the robot, which was 1587 seconds (about 27 minutes), because the operator directly selected targets more than 7 m from the robot. Moreover, we measured the success rate of supervision requests after the training phase using a data set with 52 approach and 657 non-approach targets. 4.2.1

Operating Time

Table 1 shows the measured operating time. The experimental time represents the total time of the experiment, which was 2,414 seconds. The time someone existed in the environment within 8 m of the robot was 1587 seconds and the operating time was 845 seconds. The time during which only decisions were made was 147.5 seconds. Thus, operation time accounted for 34.9% (845/2,414) of the experimental time and 53.2% (845/1587) of the time someone was within 8 m of the robot. These results indicate that the developed system successfully reduced the operation time. The average decision time was 1.83 seconds. 4.2.2

Success Rate of Supervising Requests

Table 1 shows the success rate of supervision requests. It was 86.5% for approach targets when the request timings were two seconds quicker than the operator’s decision. The supervision request rate of non-approach targets was 34.6%. These results indicate that the adaptive supervisory control system did not overlook targets to be approached and successfully reduced requests for non-approach targets.

5 Conclusion We developed an adaptive supervisory control system that enables a humanoid robot to approach appropriate people to provide information without bothering those who are not interested. The system autonomously selects a target person and requests

Adaptive Supervisory Control of a Communication Robot

563

permission to approach from the operator. Moreover, it learns from the operator decisions and improves its target selections to approach. Since the humanoid robot is mostly autonomous, the operator only decides whether to approach. After receiving permission, the robot approaches the target and provides information to that person. We conducted a field trial in a shopping mall. The results suggest that the system well worked in a real environment. The system successfully requested 86.5% of targets to approach. The operator only controlled the robot for 6% of the experiment time to decide to approach and only monitored 34.9% for safety purposes. Acknowledgements. We wish to thank the staff of the SUMISHO URBAN KAIHATSU CO., LTD. We also thank the following ATR members for their helpful suggestions and cooperation: Dylan F. Glass and Zenta Miyashita. This research was supported by the Ministry of Internal Affairs and Communications of Japan.

References 1. Wada, K., Shibata, T.: Robot therapy in a care house - results of case studies. In: Proc. IEEE RO-MAN 2006, pp. 581–586 (2006) 2. Movellan, J.R., Tanaka, F., Fasel, I.R., Taylor, C., Ruvolo, P., Eckhardt, M.: The RUBI project: a progress report. In: Int. Conf. on Human Robot Interaction, pp. 333–339 (2007) 3. Breazeal, C., Scassellati, B.: Infant-like social interactions between a robot and a human caretaker. Adaptive Behavior 8(1), 49–74 (2000) 4. Nakadai, K., et al.: Real-Time Auditory and Visual Multiple-Object Tracking for Robots. In: Proc. Int. Joint Conf. on Artificial Intelligence, pp. 1425–1432 (2001) 5. Koizumi, S., et al.: Preliminary Field Trial for Tele-operated Communication Robots. In: IEEE International Workshop on Robot and Human Communication, pp. 145–150 (2006) 6. Woods, S., et al.: Comparing Human Robot Interaction Scenarios Using Live and Video Based Methods, Towards a Novel Methodological Approach. In: Int. Workshop on Advanced Motion Control, pp. 750–755 (2006) 7. Green, A., et al.: Applying the Wizard-of-Oz Framework to Cooperative Service Discovery and Configuration. In: Proc. IEEE Int. Workshop on Robot and Human Interactive Communication, pp. 575–580 (2004) 8. Shiomi, M., et al.: A Semi-autonomous Communication Robot -A Field Trial at a Train Station. In: ACM/IEEE 3rd Annual Conference on Human-Robot Interaction, pp. 303– 310 (2008) 9. Hayashi, K., et al.: Humanoid robots as a passive-social medium - a field experiment at a train station. In: ACM/IEEE 2nd Annual Conference on Human-Robot Interaction, pp. 137–144 (2007) 10. Bergstr, N., et al.: Modeling of Natural Human-Robot Encounter. In: IROS 2008 (2008) (to appear) 11. Glas, D.F., et al.: Laser Tracking of Human Body Motion Using Adaptive Shape Modeling. In: IEEE/RSJ Int. Conf. on Intelligent robots and systems, pp. 602–608 (2008) 12. Hall, E.T.: The Hidden Dimension. Anchor Books (1990) 13. Nishio, S., et al.: Robotic Platforms Structuring Information on People and Environment. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (2008) (to appear)

564

M. Shiomi et al.

14. Jensen, F.V.: Bayesian networks and Decision Graphs. Springer, Heidelberg (2001) 15. Tajika, T., et al.: Automatic Categorization of Haptic Interactions -What are the Typical Haptic Interactions between a Human and a Robot? In: IEEE International Conference on Humanoid Robots, pp. 490–496 (2006) 16. Sugiyama, O., et al.: Three-layered Draw-Attention Model for Humanoid Robots with Gestures and Verbal Cues. In: IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pp. 2140–2145 (2005)

Behavior Design of a Human-Interactive Robot through Parallel Tasks Optimization Yuichi Kobayashi, Masaki Onishi, Shigeyuki Hosoe, and Zhiwei Luo

Abstract. Robots that interact with humans are required to achieve multiple simultaneous tasks such as carrying objects, collision avoidance and conversation with human, in real time. This paper presents a design framework of the control and the recognition processes to meet the requirement by considering stochastic behavior of humans. The proposed designing method first introduces petri-net. The petri-net formulation is converted to Markov decision processes and dealt with in optimal control framework. Two tasks of safety confirmation and conversation tasks are implemented. Tasks that normally tend to be designed by integrating many if-then rules can be dealt with in a systematic manner in the proposed framework. The proposed method was verified by simulations and experiments using RI-MAN.

1 Introduction Human-robot interaction has come to gather attention recently. [1, 2, 3]. It is also expected that robots interact with humans in household environments. Yuichi Kobayashi Tokyo University of Agriculture and Technology e-mail: [email protected] Masaki Onishi Information Technology Research Institute, AIST e-mail: [email protected] Shigeyuki Hosoe RIKEN Bio-mimetic Control Research Center e-mail: [email protected] Luo Zhiwei Kobe University e-mail: [email protected]

566

Y. Kobayashi et al.

In the environments where robots interact with humans, robots are required to do multiple tasks, such as conveyance of objects, conversation with humans, collision avoidance against humans. These parallel tasks should be sometimes done simultaneously in real time. In addition, human interacting robots are required to realize state recognition which includes uncertainty. This uncertainty is mainly caused by arbitrary motions of humans. The problem of real-time processing of parallel tasks has been discussed in the field of task scheduling [4]. In the case of multiple tasks of the human interacting robot, multiple tasks and requirement for the task are not well-defined because there have not been many trials of formulation of the parallel tasks of robots interacting with humans. One reason for this is that the uncertainty caused by human behaviors is not simple enough to formulate in a standard task scheduling framework. In the literature of robot control architectures, there have been many models to realize reactive and adaptive behaviors of robots [5, 6]. As applications of the petri-net [9], a selection framework of multiple navigation behaviors [3], a hierarchical control including exceptional handling [7], and a motion generation of a humanoid robot using timed petri-net [8] were proposed. These works mainly focused on navigation or the motion of the robot body as applications and human-interacting aspects are not considered because of the difficulty of formulation. In this paper, human-interacting tasks such as collision avoidance and conversation are implemented. The robot receives command by conversation while taking care about the safety of the human (collision avoidance). This paper proposes dealing with the design of human-interacting behavior through the modeling of petri-net, optimal control and models of human behaviors interacting with robots. The proposed architecture consists of description of parallel tasks by petri-net and transformation of the petri-net form into Markov Decision Processes (MDPs). A general formulation of the proposed design is described in 2. Later an application to parallel tasks using a human-interacting robot RI-MAN [10] is explained in 3. The experimental and simulation results are shown in 4, followed by conclusion in 5.

2 Formulation of Parallel Tasks and Optimal Control State variables express the state of the environment, human and the robot. They consist of continuous variables xc ∈ RNc and discrete ones xd ∈ ZNd . Observation variables consist of continuous variables yc ∈ RMc and discrete ones yd ∈ ZMd . The output to the actuators and the speaker consists of continuous variables uc ∈ RLc and discrete ones ud ∈ ZLd .

2.1 Description of Parallel Tasks with Petri-net The tasks are expressed first by Petri-net. Petri-net consists of places and transitions (see Fig.1). A circle on a place denotes a token. The token moves

Behavior Design of a Human-Interactive Robot Fig. 1 Example of petrinet expression for two tasks i and j

567

task j

task

i

d1j

d 2j

d1i

d 2i

place

transition

d 3j

d 0j

d 0i token

objective place

from a place to another through a transition. When a token moves, the corresponding transition is said to ‘fire’. A transition which is connected to multiple places as input can fire only when all of its input places have tokens. 1 The internal stages of task execution are expressed by places of petri-net. The number of tasks is denoted by n. An objective place di0 is defined to describe the desired stage of task i. The number of places for task i is denoted by mi and places of task i are denoted by di0 , di1 , · · · , dimi . In this paper it is assumed that there is always only one token per task. There are cases where a single place has multiple transitions as outputs. In such cases, which of these transitions will fire is characterized by firing probabilities. In addition, two assumptions are introduced; 1) an expected duration is assigned to each transition and 2) a token can stay at the same place for a certain period. 2) is expressed by defining transition from a place to itself in the process of transformation to MDPs.

2.2 Optimal Control in Markov Decision Process The stages of the task execution of the robot can be represented as s = {s(1) , · · · , s(n) }, where s(i) denotes the place that has a token in task i. This can be regarded as ‘state’ of the task execution. When an action is determined as a function of state as a = π(s), π is called a policy. Note here that the transitions of states are stochastic even for the same actions because of uncertainties caused by human motions. The transitions among the states are characterized by state transition (i) (i) (i) (i) probabilities p(sk , sl , a) and expected durations r(sk , sl , a) for transi(i) (i) tions from sk to sl . This expected duration is dependent on the action. That is, a common expected duration T (a) is applied to all transitions of all (1) (1) (n) (n) tasks as r(sk , sl , a) = · · · = r(si , sj , a) = T (a). The goal for task i is to make the token for task i reach the objective place within the shortest expected time. In the framework of optimal control with discrete state [13],this problem is to find a policy π which minimizes the ∞ (i)  (i) expected value Eπ , where rt denotes the duration of an action r t=0 t at step t in task i. The state value function is defined as the expected return: 1

In this research, we use basic definitions of transitions and omit inhibition and synchronization functions for simplicity.

568

Y. Kobayashi et al.

(i) Vπ(i) (sk ) = Eπ



∞ 



(i) (i) r(sk , sl , a)

t=0

(i)

(i)

(i)

(i)

, r(s0 , s0 , a) = 0, Vπ(i) (s0 ) = 0, s0 = di0 .

(1) The optimal state value function V ∗ (s) satisfies the Bellman equation [12]. (i)

(i)

V∗ (sk ) = min

a∈ A(s)

 (i) sl

 (i) (i) (i) (i) (i) (i) p(sk , sl , a) r(sk , sl , a) + V∗ (sl ) ,

(2)

(i)

where s = {· · · , sk , · · · } and A(s) denotes action set for state s. If the state transition probabilities and the expected durations are known, the state value (i) function V∗ (s) can be calculated by the above equation. The optimal policy for the problem can be also derived based on V ∗ (s).

2.3 Optimization of Action Parameters An action a is more concretely expressed as a sequence of continuous outputs uc or a successive discrete output ud for a certain period. The parameterization of action is generally expressed by a(θ), where θ = [α, β] in this case. In our optimal control problem, parameter θ is determined instead of deciding action a. Parameter set depends on actions, which is expressed by Θ(a). The Bellman equation with action parameterization can be rewritten as   (i) (i) (i) (i) (i) (i) (i) (i) V∗ (sk ) = min p(sk , sl , a(θ)) r(sk , sl , a(θ)) + V∗ (sl ) , (3) a∈A(s), (i) θ∈Θ(a) sl (i) {· · · , sk , · · · }.

where s = Let π(θ) denote that a policy is parameterized by θ. Note that in this research, parameter θ is adjusted for seeking optimal control while fixing policy π. The state value of task i under π(θ) satisfies the following:   (i) (i) (i) (i) (i) (i) (i) (i) Vπ(θ) (sk ) = p(sk , sl , a(θ)) r(sk , sl , a(θ)) + Vπ(θ) (sl ) , (4) (i)

sl (i) , sk , · · · }.

where s = {· · · In order to save calculation amount for finding optimal solution for θ, nominal parameter sets θ¯ are introduced. The value ¯ can be computed off-line using (4) by fixing function under the policy π(θ) (i) ¯ θ = θ. Using the values of Vπ(θ) ¯ (s), an approximation of the optimal multitask parameter selection can be done by [θ|s] = arg min θ

n 

(i) (i) Qπ(θ) ¯ (sk , a(θ))

(i)

(i)

wi Qπ(θ) ¯ (sk , a(θ)),

i=1

≡

 sl

 (i) (i) (i) (i) (i) (i) p(sk , sl , a(θ)) r(sk , sl , a(θ)) + Vπ(θ) ¯ (sl )

(5)

Behavior Design of a Human-Interactive Robot

569

and wi > 0 denotes weighting coefficient for setting priorities among tasks. By (5), an appropriate action parameter can be selected considering (approximate) optimality of the shortest time control and priorities among tasks.

3 Implementation of Human-Interacting Parallel Tasks RI-MAN was developed to realize human-interacting tasks. The robots has a speaker, two CCD cameras and two microphones on the head. The state T T variables are xc = [xT and xd = [xcom , xnh ]T, where xangle an gl e , xhuman ] denotes joint angles of the robot, xcom denotes command given by a human instructor, xnh denotes the number of humans and xhuman denotes positions of humans in the robot coordinate. The observation variables are T T T yc = [yhuman , ysound , ytac ] and yd = ycom , where yhuman denotes position of human (face) obtained by image processing, ysound denotes orientation of human who generate sound, which is obtained by sound source localization [16], ycom denotes command or conversation ID recognized by speech and ytac denotes contact information between the robot body and humans. The output variables are uc = uangle and ud = uspeech, where uangle denotes desired velocities of joint angles and uspeech denotes ID of speech.

3.1 Definition of Parallel Tasks Fig.2 shows the tasks implemented in this research. Two tasks of security of collision avoidance task and conversation task are simultaneously executed. Task 1: Security of Collision Avoidance Task The robot looks around itself and confirms whether human exists or not. When the robot judges that a human exists in vicinity, the robot speaks to human so that human does not approach to the robot any more. The robot estimates probability of human existence ph (i1 , i2 ), where [i1 , i2 ] denotes a grid that is generated by dividing the 2D space around the robot. For the judgment of human existence, two threshold values pth1 and pth2 (0 < pth1 < pth2 < 1) are introduced and used as human exists at [i1 , i2 ]

when ph (i1 , i2 ) > pth2

(6)

when ph (i1 , i2 ) < pth1 (7) human does not exist at [i1 , i2 ] human existence is unknown at [i1 , i2 ] when pth1 < ph (i1 , i2 ) < pth2 . (8) Let Rvicinity denote the vicinity of robot set and Rcontact denote the contact set. The grids are classified into three sets, Rvicinity, Rcontact and the rest Rrest = Rall \(Rvicinity ∪ Rcontact ). There are four places in the task:

570 Fig. 2 Expression of tasks by Petri-net

Y. Kobayashi et al. d 21

d11

unknown

attention

speak

found

lost

d 02

d12

d 22

conversation task d 31

safety

collision d 01

security of collision avoidance task

• ‘COLLISION’; there exists a grid [i1 , i2 ] with probability ph (i1 , i2 ) > pth2 , [i1 , i2 ] ∈ Rc ontact . • ‘ATTENTION’; the token is not at ‘COLLISION’ and there exists a grid with probability ph such that ph (i1 , i2 ) > pth2 , [i1 , i2 ] ∈ Rvicinity . • ‘UNKNOWN’; the token is not at ‘COLLISION’ nor ‘ATTENTION’, and there exists a grid such that pth1 < ph < pth2 , [i1 , i2 ] ∈ Rrest ∪ Rvicinity. • ‘SAFETY confirmed’; the token is not at other places. That is, all the grids in Rvicinity has the lower probability ph such that ph < pth1 . SAFETY is the objective place of the security task. COLLISION is judged by tactile sensor. Task 2: Conversation Task The robot faces to a human and speaks to him or her. When the robot does not receive any speech from humans, the robot promote the conversation by orienting the face to a person. There are three places in the conversation task. • ‘Human FOUND’; there exists grid [i1 , i2 ] such that ph (i1 , i2 ) > pth2 . The token on this place will transit to SPEAK when a speech is recognized. • ‘SPEAK’; based on the result of recognition of human speech, the robot outputs some reply through the speaker. The end of the output triggers the transition of the token from SPEAK to FOUND. • ‘Human LOST’; there does not exist any grid such that ph (i1 , i2 ) > pth2 . SPEAK is the objective place in the conversation task. The transition from FOUND to SPEAK depends on the utterance of human. This process is expressed by a stochastic transition.

3.2 Head Trajectory Generation through Optimization The head swinging action is executed for all places of the security task and FOUND and LOST places of the conversation task. Let ntraj denote the number of candidate trajectories. The head trajectory is generated by choosing one from ntraj candidates through optimization. To generate a candidate of trajectory θk (k = 1, · · · , ntraj ), several grids are extracted stochastically (see Fig.4). Head angles (denoted by [qpan , qtilt ]T ) that correspond to the positions of those grids are calculated. The corresponding angles of the head comprise a candidate trajectory. ph (i1 , i2 ) gives a probability to be extracted as a via

Behavior Design of a Human-Interactive Robot

571

Rext

qtilt

q2

qpan Rvicinity

Rall

q3 q1

Rcontact

robot

qpan

Fig. 3 Grid set around robot

Fig. 4 Generation of trajectory with via points

point of a trajectory. That is, grids with high ph tend to be used as via points. The selecting probability function is denoted by pselect (ph ), which is close to one when ph ≃ 1 and close to zero when ph ≃ 0. Let qi denote ith via point as qi = [qpani , qtilti ]T . By succession of those points, a trajectory is generated as θk = {q1 , q2 , · · · , qnkv },where nkvdenotes the number of via-points of candidate θk . The head angles are ordered so that arctan(qtilti /qpani ) becomes monotonically increasing. By setting the velocity of head angle vhead as constant, the total time for a periodic motion can be expressed by k

T (a(θk )) =

nv  qj+1 − qj  j=1

vhead

,

qnkv +1 ≡ q1 . (i)

(9)

(i)

This total time corresponds to the duration as r(sk , sl , a(θk )) = T (a(θk )). (1) (1) In the case of security task, the transition probability p(sk , sl , a(θk )) is calculated as followings. First, grids that are visible by a sequence of head motions generated by head trajectory θk are calculated and denoted as set Rvisible . Let Rugrid denote set of grids where humans existence is not known (defined by (8)). If view range realized by θk does not cover unknown grids, the token remains at UNKNOWN place. This can be expressed as (1)

(1)

If Rvisible ∩ Runknown = ∅, then p(s2 , s2 , a(θk )) = 1, (1)

(1)

(1)

(1)

p(s2 , s1 , a(θk )) = p(s2 , s0 , a(θk )) = 0, (10) (1)

(1)

(1)

where s0 , s1 and s2 denote SAFETY, ATTENTION, UNKNOWN, respectively. On the other hand, when Rvisible ∩ Runknown = ∅, transition probabilities are given by the followings.

(1) (1) p(s2 , s1 , a(θk )) = (1 − ph (i1 , i2 )) (11) [i1 ,i2 ]∈R vicinity visible ∩R

(1) (1) p(s2 , s2 , a(θk ))

(1)

(1)

(1)

(1)

= 0, p(s2 , s0 , a(θk )) = 1 − p(s2 , s1 , a(θk ))

(12)

572

Y. Kobayashi et al.

3.3 Model of Human Conversation It is assumed that the speech of human happens stochastically and the probability of speaking depends on how much the robot is looking at the person. The motion of the robot head is periodic with a period of T (a(θk )). The ratio of duration when the robot is facing to the human is defined as ξ(θk ) =

τ(i1 ,i2 ) (θk ) , T (a(θk ))

a human exists in grid [i1 , i2 ],

(13)

where τ(i1 ,i2 ) (θk ) denotes the duration when [i1 , i2 ] is included in the view range. The probability of human speech can be expressed as a function of ξ(θk ) as pspeech(ξ(θk )). pspeech(ξ)) is defined so that it becomes close to one when ξ ≃ 1 and decreases as ξ gets close to zero. The transition probability (2) (2) in the conversation task is expressed as p(s1 , s0 , a(θk )) = pspeech(ξ(θk )).

4 Experiment and Simulation The proposed framework is evaluated in simulation and experiment. In the evaluation, stochasticity of human behavior is omitted by 1) evaluating how the robot acts on the human instead of evaluating human utterance, and 2) fixing trajectories of humans instead of letting humans walk arbitrarily. Simulation was performed ten trials for each strategy, 300 [sec] for one trial. Two virtual humans repeat walking along fixed trajectories around the robot. Gazing time, neglecting time and average number of unknown grids are plotted in Fig.5, Fig.6 and Fig.7. Gazing time denotes the duration when the robot was looking at a human. This duration affects the probability of human utterance and thus affects achievement of the conversation task. Neglecting time denotes the duration when the robot did not notice a human who was in the region of Rvicinity . The long duration of neglecting time means that the security task is not sufficiently achieved, because a token on UNKNOWN should move to ATTENTION as soon as possible. Average number of unknown grids is calculated using (8). This number indirectly relates to the achievement of the security task because it becomes smaller when the robot keeps searching longer than tracking. When [w1 , w2 ] = [1, 0], optimization of (5) is done considering the security task only. As a result, the robot keeps searching. On the contrary, when [w1 , w2 ] = [0, 1], the conversation task is considered in the optimization. In this case, the robot keeps following humans. By changing [w1 , w2 ], the balance between searching and tracking humans can be adjusted. From Fig.5-Fig.7, it can be seen that the performance of the robot is adjusted by changing parameters w1 , w2 . In Fig.5, the gazing time was the longest in case of [w1 , w2 ] = [0, 1]. That is, the best performance of the conversation task was achieved by putting priority to the conversation task. On the contrary in Fig.7, the number of unknown grids was the largest in case of

100

neglected time [sec]

gazing time [sec]

250 200 150 100 50 0

[1,0] [1,1] [0.7,1] [0.5,1] [0,1] [w1 , w2 ]

Fig. 5 Gazing time

80 60 40 20 0

[1,0] [1,1] [0.7,1][0.5,1] [0,1] [w1 , w2 ]

Fig. 6 Neglected time

neglected time [sec]

gazing time [sec]

200 160 120 80 40 0

573

[1,0]

[0.7,1] [w1 , w2 ]

[0,1]

Fig. 8 Gazing time in experiment

# of unknown grids [sec]

Behavior Design of a Human-Interactive Robot 12 10 8 6 4 2 0

[1,0] [1,1] [0.7,1][0.5,1] [0,1] [w1 , w2 ]

Fig. 7 Unknown grids

200 160 120 80 40 0

[1,0]

[0.7,1] [w1 , w2 ]

[0,1]

Fig. 9 Neglected time in experiment

[w1 , w2 ] = [0, 1]. That is, the performance of the security task, to look around and decrease the number of unknown grids, was sacrificed by putting priority to the tracking behavior. Next in the experiment, two (real) humans walk around RI-MAN along fixed trajectories for 420 [sec] per one trial. In Fig.8, the gazing time is maximum in the case of [w1 , w2 ] = [0, 1]. In Fig.9, the neglecting time is minimum in the case of [w1 , w2 ] = [1, 0]. In the case of [w1 , w2 ] = [0.7, 1], an intermediate performance was obtained both in the gazing time and the neglecting time. Thus, a similar tendency to the case of simulation could be seen also in the experiment.

5 Conclusion This paper proposed a behavior design of a human-interacting robot which is required to execute multiple parallel tasks under uncertainties caused by humans. MDPs were constructed based on the description of parallel tasks by the petri-net. The control framework was proposed as a shortest-time optimal control problem and the multiple task problem could be dealt with in a systematic manner. In the application to the security task and the conversation task of RI-MAN, models of human behaviors were introduced. By simulation and experiment, it was verified that the proposed framework enables to adjust the performance of the robot by changing weighting parameters.

574

Y. Kobayashi et al.

References 1. Kanda, T., Hirano, T., Eaton, D., Ishiguro, H.: Interactive robots as social partners and peer tutors for children: A field trial. Human Computer Interaction 19(1-2), 61–84 (2004) 2. Shiomi, M., Kanda, T., Ishiguro, H., Hagita, N.: Interactive humanoid robots for a science museum. IEEE Intelligent Systems 22(2), 25–32 (2007) 3. Kim, G., Chung, W., Park, S., Kim, M.: Experimental research of navigation behavior selection using generalized stochastic petri nets for a tour-guide robot. In: Proc. of IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (2005) 4. Bazewicz, J.: Scheduling computer and manufacturing processes. Springer, Heidelberg (1996) 5. Brooks, R.A.: A robust layered control system for a mobile robot. IEEE Journal of Robotics and Automation RA-2, 253–262 (1986) 6. Connell, J.H.: Sss: A hybrid architecture applied to robot navigation. In: Proc. of the 1992 IEEE Conf. on Robotics and Automation, pp. 2719–2724 (1992) 7. Lehmann, A., Mikut, R., Asfour, T.: Petri nets for task supervision in humanoid robots. In: Proc. 37th International Symposium on Robotics, pp. 71–73 (2006) 8. Kobayashi, K., Nakatani, A., Takahashi, H., Ushio, T.: Motion planning for humanoid robots using timed petri net and modular state net. In: Proc. of the 2002 Int. Conf. on Systems, Man & Cybernetics, pp. 334–339 (2002) 9. Haas, P.J.: Stochastic Petri Nets. Springer Series in Operations Research (2002) 10. Odashima, T., et al.: A soft human-interactive robot ri-man. In: Video Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (2006) 11. Ramage, P.J.G., Wonham, W.M.: The control of discrete event system. Proc. IEEE 77(1), 81–98 (1989) 12. Sutton, R.S., Barto, A.G.: Reinforcement Learning. MIT Press, Cambridge (1998) 13. Bertsekas, D.: Dynamic Programming and Optimal Control. Athena Scientific (2005) 14. Elfes, A.: Using Occupancy Grids for Mobile Robot Perception and Navigation. Computer 22(6), 46–57 (1989) 15. Stepan, P., Kulich, M., Preucil, L.: Robust data fusion with occupancy grid. IEEE Trans. on Systems, Man, and Cybernetics Part C 35, 1 (2005) 16. Nakashima, H., Ohnishi, N., Mukai, T.: Self-Organization of a Sound Source Localization Robot by Perceptual Cycle. In: 9th Int. Conf. on Neural Information Processing, vol. 2, pp. 834–838 (2002)

Part VIII

Human-Robot Interaction

Tracking and Following People and Robots in Crowded Environment by a Mobile Robot with SOKUIKI Sensor Akira Ohshima and Shin’ichi Yuta

Abstract. An experimental system of the autonomous mobile robot witch follows a people or other robots was implemented and demonstrated in various situation. For this purpose, an algorithm to track an object by the moving sensor is considered and designed. In this method, SOKUIKI sensor (Scanning Laser Range Finder) output are combined with odometry data, in order to find the particular target object the others, and to estimate the target object motion. The many experimental have shown the robust target object following ability of this system.

1 Introduction Recently, many types of the service applications for the people by the robots are considered. For such applications, the robot is requested to work in the ordinary environment where people live. Especially for the mobile robots, the tasks such as going to the particular target position, or following a particular object robustly are often required to be realize in the cluttered environment with many people. In this research, we have realized a robust mobile robot motion of following a human or the other moving objects in relatively environments. The target motion, or just chasing the targets, is a simple task, and have already been demonstrated by others ([1][2][3][6][4][5]). However, most of then could not move robustly enough in the cluttered real environments, because the behaviors of the people in the environment or the objects are difficult to be modeled. In this research, an approach with the incremental design and implementation using many concrete experiments in the realistic environments are adopted to realize a real robot system which achieves such a task. Akira Ohshima and Shin’ichi Yuta Intelligent Robot Laboratory University of Tsukuba, Japan e-mail: {oosima,yuta}@roboken.esys.tsukuba.ac.jp http://www.roboken.esys.tsukuba.ac.jp/

576

A. Ohshima and S. Yuta

Fig. 1 Used mobile robot platform ”Beego” with SOKUIKI sensor

The developed algorithm is more robust and reliable compared with the former demonstrations, and a result, we have successfully conducted the experiments in the wide range of real complicated and cluttered environments. In this paper, we report on the robot hardware, algorithm and software of the implemented system, and discuss on the performance of the experimental results.

2 System Structure of the Implemented Robot 2.1 Hardware and Sensor of the Robot In this research, we considered on the wheeled mobile robot which can rotate around its the center point when it stands still. As the implementation, we used the small size mobile robot platform ”Beego”(Figure 1, [7]) which is developed by us at University of Tsukuba for the research and education. ”Beego” is 31cm(L)* 24.5cm(D)* 30cm(H) in size, and 6kgs in weight. It has two driving wheels (8.2 cm diameter) in both sides of the body, and one caster wheel (6.2 cm diameter) in rear of the body. Two driving wheels are driven by DC motors independently to make running and steering by PWS method. The mobile robot ”Beego” has incremental type rotary encoders attached to motors, to get the wheel rotation information as internal sensors. As an external sensor, SOKUIKI sensor (scanning laser range finder, Hokuyo URG-04LX[8]) is attached at the rotation center of the robot at the height of 19cm with the even level scanning plane.

2.2 Controller The mobile robot platform ”Beego” in this implementation is controlled by microcomputer (CPU: SH7045F) and laptop computer (Hardware: Lenovo X61, Software: Linux and GCC). In the micro-computer, the servo control process is working

Tracking and Following People and Robots in Crowded Environment

577

every 1 millisecond, to make rotation velocities of two motors to track the given reference value using the feedback control of rotary encoder signal. The communication between the micro-computer and the laptop computer are made every 5 milliseconds via serial communication line, in which, the reference wheel rotation velocities are given from the Laptop computer and the resultant rotary encoder signal is replied. In the laptop computer, all necessary calculation for robot control to follow the target object is performed, using the SOKUIKI sensor readings and the estimated robot position given by the accumulation of the odometry data.

3 The Realized Robot Behavior The robot is requested to follow the given people or an object in a certain degree of cluttered situation. The environment is the hallway or a room in the building such as the University or the office, in which the floor is almost flat. Because of the limitation of the robot platform and SOKUIKI sensor, we assumed the condition of the motion of target objects as less than 1 meter/second in speed and 10 meter/second2 in acceleration, which is usually fulfilled in the people’s motion. And, we also assumed that the different objects keep more than 20 cm distant between them while it moves. This condition is also fulfilled in usual situation. In this environment, the robot motion is designed to track the target object keeping less than 1 meter, so that the other object does not cross their interspaces. The realized algorithm of the robot has no shape model of target object, and the reflecting points from target object in the SOKUIKI sensor data, may not be connected. So, the robot can track and follow the objects with many shapes, such as human, handcart or the other robot. And if the several robots are set in line at first and each robot has the same program, the parade of them is easily realized, when a human lead the first robot by walking in front of it. The resultant motion is very robust and almost no failure to track the human or the other robot in the real environment, unless too dense human or the other moving objects (more than around 3 human in 1 m2 ) existing in the environment. For example, the robot can follow a people walking naturally in the 400 meters corridors of the Engineering Buildings of University of Tsukuba, without almost no failure. Figure 2 is the photograph of the demonstration of the motion, in which three robots are controlled by the same program. The first robot follows the human and the second and the third robot follow to the front robot. This motion is easily preformed because the program does not have any particular shape model of the following target and unnecessary to give any other parameters. This program and the robot motion are named ”Karugamo”(Spot-billed Duck), since this motion is associated with the natural Karugamo family, in which children follow their motion in a low. Figure 3 shows the other situation in which each of three robots follows the different person. It shows that the implemented program works in the cluttered environment with other moving objects.

578

Fig. 2 Experiment, three robots are follow to the front target

A. Ohshima and S. Yuta

Fig. 3 Experiment, three robots are follow to the different person

4 Algorithm to Track a Target Object in the Moving SOKUIKI Sensor Data In this section, the method to process the SOKUIKI sensor data in order to find the particular target object in the detected objects at every time interval, is explained.

4.1 Sensor Data Pre-processing SOKUIKI sensor measures the distance for the view angle of 240◦at each 0.36◦ , by mechanical scanning in every 100 milliseconds. So there are some time difference between the measurement and sensor data reading, and the robot motion in this time difference may not be ignored. So, the program gets the encoder counting at every 5 milliseconds and calculates the odometry-based estimated position. And, the SOKUIKI sensor data are transformed to the position Xi = (xi , yi ) in X-Y coordinate around the robot posture. Then, the isolated points from other ones in this plane are discriminated as noise, i.e. for a reflecting point i, if there is no i = j s.t. ||Xi − Xj || < ε1 , this reflecting point i remove from sensor data, where we set ε1 = 3cm. To avoid the extreme un-even distribution of the reflecting points in X-Y plane, remove the reflecting points partially from the densely existing area. That is, for all i and j (i = j), if ||Xi − Xj || < ε2 (where, ε2 = 1cm), remove one of Xi or Xj from sensor data. Next, if two points Xi and Xj fulfill ||Xi − Xj || < ε3 (ε3 = 7.5cm), then we assume that Xi and Xj are connected. So, sensor data are divided by the connectivity to C1 , ...,Cn , and we call each divided set as a cluster of reflecting points. Figure 4 shows above explained process. The each cluster can be considered as an object in environment, a part of an object, or the gathering of objects. The robot recognizes the target object to follow, as the set of these clusters. Human has two legs, and even one physical object may be

Tracking and Following People and Robots in Crowded Environment

579

Fig. 4 An example of reflecting points in pre-processing

observed as separated clusters of reflecting points because of the noise or occlusion. So, at most four clusters may be treated as the following target. The clusters got here are used to find the target object, but after deciding the following targets, only the reflecting point data of these clusters are used in control of the robot, or finding the target in the next step.

4.2 Find the Target and Estimate Their Motion 4.2.1

Initialization

The robot observes its front at the initialization step. And if an object is found in front, it is assumed to be a target to follow. In this step, the clusters of reflecting points with the condition below are regarded as the following target(Figure 5). 1. The cluster includes a reflecting point (xi , yi ) such as 0 < xi < 60cm and −20cm < yi < +20cm 2. All reflecting points (xi , yi ) in the cluster are located inside of the area as 0 < xi < 60cm and −60cm < yi < +60cm 4.2.2

Update of Reflecting Points of the Following Target

It is not easy for the mobile robot to recognize moving objects in environment, especially when the mobile robot itself is moving, because the robot sensors coordinate always moves in the global coordinate, and more over, the observed feature point of the object may also move in the object caused by the motion of the view-point. So, this is the most important part in tracking in the sensor data and following the target by the robot. In this algorithm, we estimate the motion velocity vector for each reflecting point of the following target. These velocities take an important role to decide the following target in the next sensor data. The process to decide the following target after getting new sensor data and pre-processing are given below, and schematically shown in Figure 6.

580

A. Ohshima and S. Yuta

Fig. 5 An example of reflecting points in initialization

Fig. 6 An example cluster unit of the reflecting points after update

In this step, the sensors coordinate frame is used which is the same as the robot coordinate frame {R} at the time when new sensor data are obtained. First, calculate the relationship of the coordinate frames of robot positions between previous sensor data and current data acquisition time. And, the previous following target points and their estimated velocity vectors on the previous coordinate frame are transformed to the newly acquired sensor data frame {R}. Here, let’s denote the set of previous following target points as {Sp}, and the position and velocity as Xpj and Vpj for j ∈ {Sp}. In second, consider the linear line segment X p j + k ∗ V p j ∗ Δt in which, (0 ≤ k ≤ 1, Δt = 0.1seconds). If there exists a newly acquired sensor data {Xi } (reflecting point), of which the distance to this line segment is less than ε4 = 15cm, the cluster including this point is regarded as the candidate of the following target cluster in the current sensor data set. The parameter k(0 ≤ k ≤ 1) is for the case that the following target object may suddenly reduce its speed. And the parameter ε4 = 15cm is included for the case that the following target may suddenly change its moving direction. The candidate of the following target cluster of reflecting data may include not only the real following target but also the other objects nearby the target. And, in

Tracking and Following People and Robots in Crowded Environment

581

the case that the false object is a big one like a wall in the environment, it should be omitted from the candidate. So, in third, if a cluster CR includes existing sensor reflecting point Xi which is distant more than 30cm to line segment X p j + k ∗V p j ∗ Δt for all j ∈ {Sp}, the cluster CR is judged to be omitted from the candidate. And the remaining clusters are regarded as the cluster of reflecting points for the following target. 4.2.3

Judgment of Failure

The above mentioned process can usually give the target object to follow by the robot in the newly acquired sensor data. However, since treating the real sensor with uncertainty, the perfect recognition of the object is impossible and there may exist a fatal failure to find the target object. The first case is the losing track of the object. This case may be occur when the tracking object jumped to side. And, the second case is the detecting and selecting the other objects as the target. This may be occurring when the environment is too cluttered, or the following target moves too near to the other object. Even though the robot may not continue the target tracking any more, it is still important for the autonomous robot to recognize the failure. For the judgment of this situation, robot checks the following condition. And if one of these conditions meets, it recognizes the failure of tracking. 1. No clusters remain in the newly acquired sensor data 2. More than five clusters are regarded as the following target 3. The size of the following target exceeds 150cm Let’s denote the set of resultant reflecting points which is recognized to belong the following target by {S}. 4.2.4

Velocity Vector Estimation on Each Reflecting Point

The reflecting points of the following target are recognized as mentioned at 4.2.2. For continuing to recognize the reflecting points in the next observed sensor data, each recognized reflecting point Xi ∈ {S} should have an estimated velocity vector. For the estimation, we assume that each following target point Xi ∈ {S} comes from some of reflecting points in the previous sensor data Xpj ∈ {Sp}. And we calculate the velocity vector of each reflecting point from the weighted average of the distance between new and previous reflecting point at coordinate {R}, (Figure 7) i.e., ∑ j∈{Sp} wi j ∗ (Xi − Xpj )/Δt VFi = (1) ∑ j∈{Sp} wi j where, as the weight, we use likelihood index between Xpj and Xi wi j = f (|Xpj + Vpj − Xi |) with monotone decreasing non negative function

(2)

582

A. Ohshima and S. Yuta

Fig. 7 Calculated from the weighted average of the difference between related reflecting points

f (d) =



exp(−300 ∗ d 2) · · · 0 ≤ d ≤ 0.2 0 · · · d > 0.2

(3)

However, the set of reflecting points in the previous sensor data {Sp} is in the limited region and then the above average vector tends to direct outer side at the boundary. For compensation of this phenomenon, we decide a particular corresponding previous reflecting point Xpj ∈ {Sp} for each new reflecting point Xi ∈ {S}, s.t. Xpj + Vpj is the nearest to Xi , and the calculate converse velocity vector for the Xpj . ∑i′ ∈{S} wi′ j ∗ (Xi′ − Xpj )/Δt VBi = . (4) ∑i′ ∈{S} wi′ j Since the set of reflecting point {S} is limited, the velocity vector VBi tends to direct to inside of the region at the boundary. So, we may use the average Vi =

VFi + VBi 2

(5)

as the estimated velocity vector of the reflecting point Xi .

5 Control of the Robot By the sampling rate of SOKUIKI sensor, the target following program of the robot can obtains the set of the reflecting points of the target object to follow, at every 100 milliseconds. And it calculates the point of chasing target in the X-Y plane using the reflecting points set in this rate. It calculates a suitable rotational velocity of the robot to control the heading direction to the chasing target points, and also it running velocity to keep a constant distance (0.4 meters) to the point at same time. In case when some objects, which is not recognized as the target, is detected in the sensor data on the robot path to

Tracking and Following People and Robots in Crowded Environment

583

Fig. 8 An example the robot’s behavior toward following target

the target point, the heading direction is controlled with modification to avoid the collision. The calculated rotational velocity and running velocity is given to the vehicle control part of the system every 100 milliseconds. The vehicle control part modifies these target velocity is to meet with the physical limitations, such as maximum ˙ max = 180◦ /sec2 , velocities (ωmax = 100◦/sec, vmax = 60cm/sec) and accelerates (ω v˙max = 100cm/sec), and calculate the reference rotational velocities of two motors and gives to the micro-computer to control them. Figure 8 shows example the robot’s behavior.

6 Implementation of Software We implemented the algorithm given on the mobile robot platform ”Beego”. The program is written in C-language and its size was 900 steps. In this program, the robot initializes to find the chasing target at first, when a button of computer is pushed, and starts to track the target object, by updating the target object in the sensor data at every 100 milliseconds, unless the robot recognizes the failure of the target tracking. The necessary processing times are 5-7 milliseconds for the pre-processing sensor data, about 10 milliseconds for updating the tracking target, and about 5 milliseconds in decision of control, on the laptop computer of the robot. This processing time is small enough comparing the 100 milliseconds of sensor data sampling time.

7 Conclusion This paper explained an example of the mobile robot’s target following motion, using a simple SOKUIKI sensor.

584

A. Ohshima and S. Yuta

The algorithm in this program is very simple but carefully designed and implemented. So, as the result, the robot can follow the human or the other object robustly in the relatively cluttered environment. The following an object is one of the fundamental motion of the mobile robot, and the robustness in the various the environment condition is most important. In the implemented program, the robot can perform this motion without any preparation of the special parameter of the target. So, it is easy to demonstrate in any place or to use as a part of the other program. We think that this program could realize certain robustness in the usual crowded indoor environment. However, the environment is still limited to not too crowd. So, we will make many experiments more and analyze the more error cases, and improve the algorithm to make it more robust.

References 1. Kobilarov, M., et al.: People tracking and following with mobile robot using an omnidirectional camera and a laser. In: Proceedings of the 2006 IEEE Conference on Robotics and Automation (ICRA 2006) (2006) 2. Yoshimi, T., et al.: Development of a Person Following Robot with Vision Based Target Detection. In: Proceedings of the 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2006) (2006) 3. Chen, Z., Birchfield, S.T.: Person Following with a Mobile Robot Using Binocular Feature-Based Tracking. In: Proceedings of the 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2007) (2007) 4. Ono, Y., et al.: Development of Leader-Following Control System Development of SemiAutonomous Mobile Robot(2). In: The 25th annual conference of the robotics society of Japan (RSJ 2007), pp. 10–26 (2007) 5. Masuda, K., Fukase, Y., Kanamori, H.: Developing a Tracking Control Method of Robotic Wheelchair. In: The 25th annual conference of the robotics society of Japan, 2E-23 (RSJ 2007) (2007) 6. Nagumo, Y., Ohya, A.: Human following behavior of an autonomous mobile robot using light-emitting device. In: 2001 JSME Conference on Robotics and Mechatronics, 2P1-H6, Japan (ROBOMEC 2001) (2001) 7. Kamimura, S., et al.: The Portable Mobile Robot Project for research and education. In: The 22nd annual conference of the robotics society of Japan, 1A-33 (RSJ 2004) (2004) 8. Kawata, H., Mori, T., Yuta, S.: Design and Realization of 2-Dimensional Optical Range Senser for Environment Recognition in Mobil Robots. Journal of Robotics and Mechatronics 17(2), 116–120 (2005)

Author Index

Alami, Rachid 335 Alili, Samir 335 Aonuma, Hitoshi 97 Appelqvist, Pekka 523 Arai, Tamio 73, 465 Araki, Hirohisa 27 Asama, Hajime 97 Baglietto, Marco 3 Bekey, George A. 261 Bordignon, Mirko 379, 441 Brandt, David 441 Bribiescas, Carlos 355 Butler, Simon 533 Butler, Zack 355, 417 Cannata, Giorgio 3 Capezio, Francesco 3 Casalino, Giuseppe 315 C ¸ elikkanat, Hande 215 Chiba, Ryosuke 465 Chirikjian, Gregory S. 273 Chiu, Harris Chi Ho 429 Christensen, David Johan 379, 441 Chung, Jae Heon 109 Chuy Jr., Oscar 345 Clark, Christopher M. 261 Collins Jr., Emmanuel G. 345 D’Angelo, Antonio 85 Demiris, Yiannis 533 Dong, Haiwei 295 Elomaa, Mikko Endo, Mitsuru

523 305

Fabricant, Eric 417 Fujii, Norisuke 325 Fujisawa, Tomoharu 513 Fukuda, Toshio 51, 123 Funato, Tetsuro 85 Hada, Yasushi 63 Hagita, Norihiro 555 Halme, Aarne 523 Han, Chang Soo 109 Hayashi, Akinobu 73 Hino, Hisato 513 Hirata, Yasuhisa 305 Hirose, Kenji 305 Hoshino, Satoshi 503, 513 Hosoe, Shigeyuki 565 Ikemoto, Yusuke 97 Inou, Norio 393 Inoue, Reiko 325 Ishiguro, Akio 193 Ishiguro, Hiroshi 555 Jayasuriya, Suhada

477

Kaelbling, Leslie Pack 367 Kamimura, Akiya 405 Kanbayashi, Takashi 305 Kanda, Takayuki 555 Kawabata, Kuniaki 63 Kimura, Ayako 227 Kimura, Hitoshi 393 Kitamura, Taichi 193

586

Author Index

Kobayashi, Ryo 193 Kobayashi, Yuichi 565 Koseki, Michihiko 393 Kosuge, Kazuhiro 305 Kurokawa, Haruhisa 405 Lagoudakis, Michail G. 491 Leidi, Michele 543 Li, Bin 453 Liu, Xiuwen 345 Lochmatter, Thomas 239 Luo, Zhiwei 295, 565 Ma, Shugen 453 Martinoli, Alcherio 203, 239 Maruyama, Shigehisa 513 Mastrogiovanni, Fulvio 161 Mat´ıa, Fernando 203 Matusiak, Marek 523 McLurkin, James 15 McMillen, Colin 251, 283 Miyake, Yoshihiro 135 Mondada, Francesco 543 Montano, Luis 491 Montreuil, Vincent 335 Morton, Ryan 261 Mosteo, Alejandro R. 491 Murakami, Kazunori 305 Murata, Satoshi 405 Nagao, Takashi 97 Naka, Yuji 503 Nakagaki, Toshiyuki Nakamura, Kenichi Navarro, I˜ naki 203 Nikolaidis, Stefanos Nishi, Ryosuke 227 Nishinari, Katsuhiro Nohara, Kenta 555 Nojima, Mitsuharu

193 305 73

Paanaj¨ arvi, Janne 523 Pagello, Enrico 85 Pugh, Jim 203 Rey, Fran¸cois 543 Rozinat, Anne 251 Rubenstein, Michael Rus, Daniela 367

429

S ¸ ahin, Erol 215 Schindelhauer, Christian 39 Schultz, Ulrik Pagh 379, 441 Seki, Hiroya 503 Sekiyama, Kosuke 27, 51, 123 Sgorbissa, Antonio 3, 161 Shaikh, Danish 379 Shen, Wei-Min 429 Shi, Dongqing 345 Shimo, Hiroaki 135 Shiomi, Masahiro 555 Simetti, Enrico 315 Stoy, Kasper 379, 441 Sugahara, Yusuke 305 Sugizaki, Ryuji 63 Suzuki, Koki 305 Suzuki, Tsuyoshi 63 Suzuki, Yosuke 393 Takahashi, Junji 51 Takase, Kunikatsu 149 Takeda, Koichi 193 Tobe, Yoshito 63 Tomita, Kohji 405 Tomoeda, Akiyasu 227 Turetta, Alessio 315 Turgut, Ali Emre 215 Ueda, Ryuichi 73 Umedachi, Takuya 193

175, 227 135

Ohshima, Akira 575 Onishi, Masaki 565 Ooga, Jun’ichiro 149 Ooi, Chia Ching 39 Oomoto, Mitsukazu 305 Ota, Jun 325, 465, 513 Ozaki, Fumio 149

Vainio, Mika 523 van der Aalst, Wil M.P. 251 Varshavskaya, Paulina 367 Veloso, Manuela 251, 283 Wakuda, Yuki 123 Wang, Yuechao 453 Winfield, Alan F.T. 185 Yanagisawa, Daichi Yano, Shiro 97

227

Author Index Ye, Changlong 453 Yi, Byung-Ju 109 Ylikorpi, Tomi 523 Yuta, Shin’ichi 575

587 Zaccaria, Renato 161 Zhang, Junjie 477 Zhang, Kai 345 Zickler, Stefan 251